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I. INTRODUCTION 
A. BACKGROUND 


"It is a well-established rule of scientific investigation that the first 
time an experiment is performed the results bear all too little resemb- 
lance to the 'truth' being sought" [Ref. 1: p. 1]. The experiment may 
be the simple task of data collection, i.e. survey, or a process of 
generating data. The analyst may have a small set or a large.set of 
data or a series of observations which must be analyzed. After some 
data analysis, the analyst may extract quantities relevant to purposes 
that he/she has in mind for further analysis. This analysis and data 
extraction process has the formal name of data reduction.  Tukey calls 


this process "exploratory data analysis" [Ref. 2: p. 1]. 


There are several statistical methods that can facilitate the data 
reduction process. The quote, "a picture says a thousand words," 
suggests that the data analysis involves pictorial representations of the 
observed data. The single, most powerful statistical tool is a "well- 
chosen graph" [Ref. 3: p. 1]. A well-chosen graph enables salient 
features of a data set to be picked out and vividly portrayed so that 


the analyst can spot the features of particular interest [Ref. 4: p. 41]. 


The data set is very often bivariate data, i.e. pairs of values 
(е, 06 КАУ (Хүү, Хм), where it is conventional that the non 
called the ordinate, be a function of the corresponding Do called the 
abscissa. The abscissa indicates a specific snapshot of time or is the 
input to an experiment, i.e. the value of an independent variable. The 
analysis of the data basically concentrates on finding a relationship 
between the Хү апа ће Үү. The single most powerful statistical tool 
for analyzing the relationship between the Хү апа the Y, is the scatter- 
plot [Ref. 3: p. 75]. A scatterplot is a two-dimensional graph which 
visually displays the relationship of the pairs of Ат апа (1. Ihe 


vertical axis of the scatterplot represents the scale values of the 
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ordinate or ie and the horizontal axis represents the scale values of 
the abscissa ог Ху. A _ scatterplot is easily accepted by the human 
brain which quickly summarizes the depicted information and extracts 
the salient features, patterns, and relationships that are not detected 
with other data analytical methods, e.g. tabulated data. Figure 1.1 is 
an example of а  scatterplot displaying the Daily Sea-Surface 
Temperature for 1971 at Granite Canyon, just South of Point Sur, 
California [Ref. 4]. This sea-surface temperature data for 1971 is 


given in tabular form in Appendix D. 


DAILY SEA SURFACE TEMPERATURE 
AT GRANITE CANYON, CA. 
MARCH 1, 1971 TO DECEMBER 31, 1971 
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Figure 1.1 Scatterplot of Sea Surface Temperatures for 1971. 


The scatterplot shown in Figure 1.1 is more compact and informative 
than the corresponding tabulated data in Appendix D. The scatterplot 
indicates that the sea-surface temperature varies with the time of year, 
i.e. general temperatures increase during the summer and decrease 
during the fall. There may have been other extraneous factors that 
affected the temperatures, e.g. the warm ocean current El Nino, an 
intra-yearly occurrence which sometimes causes great climatic turbulence 
all over the world, could be the cause of the great temperature varia- 


tions shown by the scatterplot in Figure 1.1. It is very difficult to 
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DAILY SEA SURFACE TEMPERATURE AT GRANITE CANYON, CA. 
MARCH 1, 1971 TO DECEMBER 31, 1971 
USING SUPERSMOOTHING 
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Figure 1.2  Smoothed Sea Surface Temperatures for 1971. 


interpret the function imbeded in the scatterplot of Figure 1.1. Xm 
attempt to sketch a rough line that follows the curvature of the points 
may result in a tenuous and perhaps incorrect line in terms of depicting 
the variability such as the periodicity in the data set. The sketching 
of the line through the scatterplot of Figure 1.1 would take time and 
involve strong subjective decisions. The result could be a misinterpre- 
tation of the scatterplot/data. A more effective and substantiated 
method of data reduction is "smoothing", see Figure 1.2. This scatter- 
plot with a smooth curve through the raw data is more acceptable to the 
human eye than a plain, data scatterplot and fairly well approximates 
the raw data. A cyclic change of the sea surface temperature is 
emphasized by the smooth curve. In addition, Figure 1.2 depicts that 
a cycle with roughly a monthly period could exist in the data, i.e. 


there are twelve peaks shown by the smooth curve. 


Smoothing can be used on data sets whose scatterplots indicate an 
underlying relationship that is either a simple linear function or a 
complex sinusoid function. Smoothing has in the recent past years 
become a useful data reduction technique. Banks, insurance companies, 


and industrial firms smooth economic surveys [Ref. 5: p. 1]. 
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The government does smoothing on income data such as tax payments, 
salaries and benefits to civil servants, and social services costs 
[Ref. 6]. The space program does smoothing of test flight paths, fuel 
usage, and orbiting ejection path data [Ref. 7]. Conferences have had 


smoothing as the sole subject of discussion [Ref. 8]. 


The smoothing algorithm that is used to smooth a data set must use 
a procedure that is flexible enough to discover trends in the data, i.e. 
be able to accurately trace the observed data and respond to local 
changes. Therefore, the algorithm should use local smoothing rather 
than global smoothing which is used in linear regression and curve 
fitting. This procedure allows the observed data to determine the 


shape of the smooth curve. 


An advanced smoothing algorithm must be more computationally effi- 
cient and more user friendly than most current smoothing algorithms. 
In addition an advanced smoothing algorithm must be able to correctly 


extract the underlying function from the observed data. 
БН SCOPE 


This paper discusses and analyzes two advanced smoothing algo- 
rithms, the Supersmoother algorithm [Ref. 9] and the Split Linear Fit 
Algorithm [Ref. 10]. These two smoothing algorithm were developed at 
Stanford University and thus have many similarities. The basic concept 
used in these algorithms is that the underlying function is thought of 
as a low frequency signal; therefore, the observed raw data is the 
Signal plus noise. Thus, the smoother is analogous to a low-pass filter 
which is designed to compromise between the signal extracted, i.e. 
desirable effects, and the noise filtered out, i.e. undesirable effects 
[Ref. 10: p. 1]. Equation 1.1 shown below is a generalization of the 


low-pass filter: 


Yi= f(¥) +r, , (ШЕ) 
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were ҮТ is the observed value, 4049 is the smooth function or 
extracted signal, and ry, is the additive residual or noise filtered out. 
It is initially assumed that the set of Y, is an independent and identi- 
cally distributed (i.i.d.) random sample from some unknown joint distri- 
bution F(X, Y). It is also sometimes assumed that the r, are i.i.d. 


2, but possibly correlated; 


with zero expectation and constant variance g 
thus the notation follows the convention set by time series theory 
[Ref. 3: p 246]. The computed smooth values are estimates of the 
smooth function (Үү). It is best that the smooth point values be 
computed using local averaging [Ref. 10: p. 3], in other words the pth 
smooth point value is the average of the Y values corresponding to the 
X values within a neighborhood of size K about Xj, where a neighbor- 
hood of size K about Хү will have (K/2) point values to the right and 
left. Equation 1.2 shown below states this averaging procedure in 
conditional form, indicating that only the Y values that correspond to 
the X values within the neighborhood K about Xj are involved in the 


averaging. 
5(Х1) » average(Y ; given X, a member of the neighborhood Kj), (1.2) 


where s(Xy;) is the computed smooth point value corresponding to Xi, 
К 15 the neighborhood size corresponding to Apo J21, . . ., Ky ISBN 
Jt) member of the neighborhood of size Кү, апа I-1, . . ., N is the 
index of the N points to be smoothed. For the simple, equal-weight, 
moving average smoother, the smooth value at point Aj is computed by 


equation 1.3: 


5(Ху) = =x Ит (1-33 


where K is the neighborhood size and may encompass a fraction of the 
data set to be smoothed or the entire data set. By looking at equation 
1.3, it can be deduced that when I=1, ., (K-1) and I=(N-K-1), 


., N the subscript of Y is negative and has no corresponding Y values. 
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Most simple moving average smoothers do not involve the latter 
mentioned index values and begin the averaging with I=(K/2) and end 
the averaging with I-N-(K/2); thus, the smooth output wil have less 


values than N, exactly K less values. 


The neighborhood size, denoted above by K, referred to later in 
this thesis as bandwidth, span, or windowsize, is a critical value which 
must be chosen careful because it determines to a great degree, the 
goodness of fit of the smooth curve to the raw data. For example, with 
the equal-weight, moving average smoother, a large neighborhood size 
results in the loss of many smooth point values, and thus, the raw data 
is not well depicted. A commonly used measure of goodness of fit is 
the sum of squared residuals; thus, -it is necessary to examine a 
squared residual value in general terms. If the output of the smooth 
function 40,49 is accepted as an estimate of the corresponding Үү апа а 
linear fit is done on the points within the neighborhood, then the 
expected squared residual at point Xj, given a neighborhood size K, 


may be determined by equation 1.4: 


2 


1-— 


пра | к) = |1 (9 - x Df (Ya) + ехо? . (1.4) 


ue А. 
2 


The term within the brackets is the bias component of the estimated 
residual value corresponding to кү in other words, the degree to 
which the smooth point value deviates from the actual point value. The 
second term is the variance component which indicates that the assumed 
inherent constant variance of the residuals must be equally shared by 
the estimated residuals within the neighborhood. Increasing the neigh- 
borhood size, K, increases the bias and decreases the variance, thus a 
plot of the smooth values will get smoother as K 1$ increased. 


Decreasing K will have the opposite effect. 


Most smoothing algorithms use only one neighborhood size to 


produce the smooth values, i.e. the same K for all Xy in equations 1.3 
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and 1.4. The problem with this method is that the smoothing program 
may have to be run several times with each run containing a different 
K before the desired smoothing effect is produced. This thesis 
discusses two advanced smoothing algorithms which deviate from this 


procedure. The two advanced smoothing algorithms are: 


1. the Supersmoother algorithm developed by Friedman and Stutzle 
[Ref. 9]; 

2. the Split Linear Fit algorithm developed by McDonald and Owen 
[Ref. 10]. 


The Supersmoother requires that the user enter three different 
neighborhood sizes, SPAN), SPAN», and SPAN, in increasing order. 
Each span value determines a neighborhood size about each Xj on which 
a linear regression is done. Therefore, three sets of regression results 
wil correspond to each Xj. Each of the three slope values, the three 
corresponding y-intercept value, and the corresponding Хү are used to 
compute three fitted values. Each fitted value is subtracted from the 
input І value corresponding to М the resulting values are called 
cross-validated residuals. The minimum, absolute value of these cross- 
validated residuals is then selected along with its span value. This 
span value is an estimate of the optimal span value corresponding to 
Хр. This estimate is then adjusted using an outlier rejection rule which 
will reflect the degree of robust smoothing desired by the user. The 
smallest span value, SPAN,, and the largest span value, SPANGS dictate 
the range within which Supersmoother finds the optimal span value. 
The middle span value, SPAN», is used as a central smoother, i.e. by 
smoothing the array of optimal span values with the middle span value 
the variability is reduced. This smoothing adjusts the span values so 
that the values flow smoothly from one point to the next adjacent point. 
This method of finding the optimal span values is called local cross- 
validation [Ref. 9: p. 1]. The method of cross-validation is a testing 
procedure that uses the estimated regression equation on data different 
than the data used to estimate the coefficients of the estimated regres- 


sion equation [Ref. 11: p. 110]. 
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The Split Linear Fit algorithm uses one or more neighborhoods, 
called window sizes, in order to produce for each I a family of linear 
fitted values. Weights which indicate the goodness of fit of the fitted 
values are then assigned to each of these linear fitted values. The 
final IP 


fits within the I‘ family of linear fits. 


smooth value is computed as a weighted average of the linear 


This technique of using more than one neighborhood size allows the 
analyst to set upper and lower limits on the neighborhood size. By 
accepting more than one neighborhood size, these advanced smoothers 
take full advantage of the powerful computational capabilities of a 
computer and thus are quicker and more efficient than other smoothers, 
i.e. desired smoothing effects are achieved in less runs of a smoothing 


program. 


The purpose of this thesis is to expand the data smoothing subrou- 
tine developed by Friedman and Stuetzle [Ref. 9] and the smoothing 
program developed by McDonald and Owen [Ref. 10] into user friendly, 
interactive computer programs, i.e. the user exchanges information with 
the computer, that can be used as an exploratory data analytical tool 


by students and faculty of the Naval Postgraduate School. 


The Supersmoother algorithm was written as a FORTRAN subroutine 
and has been incorporated into an interactive FORTRAN program. The 
Split Linear Fit algorithm was part of a data smoothing package written 
in the C computer language, which is not a common computer language 
used at the Naval Postgraduate School. The Split Linear Fit algorithm 
has been translated and is incorporated into an interactive FORTRAN 
program. The point values produced by the Split Linear Fit FORTRAN 
version are equivalent to the point values produced by the C language 
version. Both the Supersmoother and the Split Linear Fit algorithms 
are written in FORTRAN 77 for use on the IBM 3033 computer being 
used at the Naval Postgraduate School. SUPSMO is the Supersmoother 
program and SPLITSMO is the Split Linear Fit program. These two 
FORTRAN programs are designed to produce output in any one of 


following three forms: 


Ша 


i a CMS data file; 
an APL, "A Programming Language, ' 
3. graphs produced with the IBM GRAFSTAT! statistical graphics 
package [Ref. 12]. 


These programs are written for use by any individual who has access to 


' variable; 


the IBM 3033. With simple commands the user can create or access an 
APL workspace and create an APL variable that stores the smooth 
output. Access to the GRAFSTAT graphics package is easy and done 
without exiting the smoothing program. Creation of a CMS file is even 
easier. GRAFSTAT is a graphics package which is an experimental 
program available at the Naval Postgraduate [Ref. 12]. 


Complete user instructions on how to use SUPSMO and SPLITSMO 
are available in Chapter VI and VII. Mathematical details on the the 
Supersmoother and the Split Linear Fit are presented in Chapters II 
and III, respectively. In Chapter IV are the evaluation results from 
smoothing three simple sets of data with these two advanced smoothers. 
These smoothing results are compared to the smoothing produced by 
previously verified smoothers, e.g. LOWESS and Moving Average. In 
Chapter V a real application of the Supersmoother and the Split Linear 
Fit programs is presented. The Granite Canyon Daily Sea-Surface 
Temperature data for the period of March 1971 to February 1983 is used 
in the analysis presented in Chapter V. This data set is used because 
of the large size of the series, 4380 points; because the variance may 
not be constant, and because the complex underlying function seems to 


contain some periodicity. 


IGRAFSTAT is an experimental APL package from IBM which the 
Naval Postgraduate School is using under an agreement with the IBM 
Research Center, Yorkstown Heights, N. Y. 
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Il. TECHNICAL DESCRIPTION OF SUPERSMOOTHER ALGORI 


— À— за 


iM 
A. OVERVIEW 


The data smoothing algorithm, Supersmoother, was developed at 
Stanford University by Jerome H. Friedman and Werner Stuetzle 
[Ref. 9]. The smoothing technique uses local averaging [Ref. 9: p. 
3], local linear fitting [Ref. 9: p. 3], and selection of a local optimal 
span [Ref. 9: p. 8], i.e. application of method of cross-validation 
[Ref. 9: p. 1]. The developers claim that Supersmoother is "both very 
flexible and rapidly computable" [Ref. 9: p. 3]. One of the features 
which makes Supersmoother flexible is that Supersmoother is scale inde- 
pendent. In other words, the X values must be equi-spaced but can 
belong to the interval (0.0, 1.0] or the interval [1.0, 2.0, 3.0, A, 
N], where N is the number of point values to be smoothed, while the Y 
values must be real values and need not be equi-spaced. Another 
feature which makes Supersmoother flexible is that there is an option of 
entering one or three global span values where these values are entered 
as a ratio of the span to the number of points to be smoothed. 
Another flexibility feature is that there is an outlier rejection rule 
which allows the user to adjust the degree of robustness using an index 
within the interval [0.0, 10.0], where 0.0 indicates robust smoothing 
and 10.0 indicates non-robust smoothing.  Supersmoother uses a small 
amount of computer time and of storage space by using computation and 
data storage procedures commonly used in dynamic programming, i.e. 


Fy-4 (4) is used to update Е1(Х) and only the new value is stored. 


The objective of Supersmoother is to efficiently smooth a scatterplot 
[Ref. 9: p. 1]. Supersmoother consists of two subroutines, the 
Combining Subroutine and the Smoothing Subroutine, see Figure 2.1. 
The Combining Subroutine and the Smoothing Subroutine exchange data 
arrays once if only one span value is used and eight times if three 


span values are used. 
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Combining 
Subroutine 


Smoothing 
Subroutine 





Figure 2.1 Supersmoother Subroutines. 


Figure 2.2 shows the flow of data within the Supersmoother algo- 
rithm. The Combining Subroutine receives the data to be smoothed and 
other pertinent parameters and sets up the data for transmission to the 
Smoothing Subroutine. The Smoothing Subroutine smoothes the data 
array three times, using each span value once, and then computes the 
residual values corresponding to the three smoothed arrays. Then each 
array of residual values is smoothed using SPAN» in order to reduce 
the total variability and create smooth transitions between adjacent resi- 
dual values. The smoothed residual values are then returned to the 
Combining Subroutine where the optimal span values are determined and 
adjusted using the outlier rejection rule. The adjusted optimal span 
values are then sent back to the Smoothing Subroutine for smoothing 
with SPAN». This is done so that variability between the values will 
again be reduced. The now smoothed, adjusted, optimal span values 
are returned to the Combining Subroutine where they are used in an 
interpolation procedure. The results of this interpolation procedure are 
estimates of the final smoothed values. These estimated smoothed 
values are then returned to the Smoothing Subroutine for smoothing 


with SPAN, in order to reduce the variability of these values. This 
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procedure will also accentuate outliers in the raw data because of the 
small neighborhood size of SPAN,. Тһе final smoothed values are 
returned to the Combining Subroutine which forwards the results to the 


user's main program for his/her use. 


The Combining Subroutine does the following: 

1. | keeps track of pertinent computational results; 

computes the interquartile range of the abscissa; 

defines zero for computational and comparative purposes; 


determines the optimal span corresponding to each abscissa; 


льо го 


applies the outlier rejection rule in order to adjust the 
robustness; 

6. estimates the smoothed output. 
If only one span value is used, only the first three items of the above 
list are executed by the Combining Subroutine, and the Smoothing 
Subroutine is used only once. If three span values are used, all the 


items are executed. 


The smoothed output produced by the Smoothing Subroutine is not 
the final smoothed values given to the user. ‘Therefore, to be able to 
distinguish the output forwarded to the user, i.e. the smoothed Y 
values, from the smoothed values exchanged between the subroutines, 
any array to be smoothed by the Smoothing Subroutine, e.g. the resi- 
dual values, will be called Z within the Smoothing Subroutine. 7. 
After the array is smoothed and returned to the Combining Subroutine, 
it regains it's usual name. The Smoothing Subroutine does the 
folowing: 

1. computes the neighborhood size, (IT), the number of points to 

be included in the local averaging; 

2. computes the base mean, variance, and covariance values that 

will be used in the computation of the smoothed values А 

3. computes the smoothed values Z, at the beginning of the data 

array, i.e. the first (IT/2) smoothed points that are not 


usually computed by most smoothers; 
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Figure 2.2 Data Flow of Supersmoother. 


4. computes the smoothed values Z, for the middle of the data 
array; 

5. computes the smoothed values Z, for the end of the data set, 
i.e. the last (IT/2) smoothed points that are not usually 
computed by most smoothers. 

The smoothed values 2] are the result of a local linear fit within the 
neighborhood of points about Xj, the abscissa corresponding to the 
smoothed 21]. The Smoothing Subroutine computes the cross-validated 


residuals only at the time when the input data is smoothed, see Figure 
252. 


B. MATHEMATICAL DETAILS---COMBINING SUBROUTINE 


The Combining Subroutine requires the following user input: 


1.  N, the number of points to be smoothed; 

2. Yi, . . ., Yy, the point values that need smoothing; 

ШЕ... Хы, the abscissa corresponding to the Y, values if 
the abscissa do not belong to the interval [1.0, 2.0, s 
N]; 

4.  IPER, equals 1 or 2, to indicate that the abscissa belong to the 
interval [1.0, 2.0, . . ., N] or the interval (0.0, 1.0], 
respectively; | 


the span values SPAN), SPAN), and SPAN; 
ALPHA, the outlier rejection rule index. 
The Smoothing Subroutine assumes that the input data set is in chrono- 


logical order , i.e. Y, occurred before Yri,, where I-1, ANS D; 


thus, the abscissa will be in increasing order. 


The abscissa interquartile range, SCALE, is computed using equa- 
tions 2.1 through 2.3: 


№. CD 


(22) 


27 


SCALE = XJ = Xr р (23) 


If N<4 and if the computer allows indices with value of zero, then 
SCALE=0.0, otherwise an error is created. In order to define zero, 
VSMLSQ, SCALE must be greater than zero. If SCALE<O.0, then J=J+1 
and SCALE is recomputed using equation 2.3. Zero is defined by 


equation 2.4: 
VSMLSQ = [(1x10 3) SCALE? . (2.4) 


If only one span value is used, the Smoothing Subroutine is called 
by the Combining Subroutine, and the smoothed data array returned is 
the smoothed У], see Figure 2.1. If this procedure is used, the 
smoothed Yy wil have too much variability [Ref. 9: p. 9] and wil be 


very robust, so it is best to use three span values. 


When three span values are used, the input i are smoothed three 
times, once with each span value, see Figure 2.2. For ease of discus- 
sion, Yrs wil be used to indicate the jth input Y value smoothed using 
SPANG, where I-1, ., N and S-1, 2, 3. As mentioned before, 
during the smoothing of the input ҮТ, cross-validated residual values 
are computed. These residual values will be identified by ACVRic, 1.6. 
the It8 cross-validated residual computed when SPAN was used. In 
order to reduce the variability of the smoothed Үт, Ше АСУК а аге 
smoothed using SPAN», see Figure 2.2. For stability reasons an array 
containing the absolute value of the ACVRjgs is smoothed. [Ref. 9: p. 
9]. After the smoothing of the absolute value of the ACVR[is, each 
abscissa ХТ, has the following seven corresponding values: 

1. the input Yi; 

2 Yy, and ACVRy,; 

3. Үр апа АСУК1; 

4 Үтҙ апа ACVR1;. 


Next follows the basis of the local cross-validation method. First 


for each I the minimum of ACVR;, ACVRy5, and ACVR, is selected and 
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designated ACVR,.,. Recalling that the second subscript of ACVRis 
indicates the SPAN used to smooth the input data set and produce the 
corresponding ACVRis, then the SPANS used to produce АСУК іп сап 
be determined and designated SCr. Then the outlier rejection which 
consists of equations 2.5 and 2.6, shown below, can be used to adjust 
SCy, i.e. the span value used to compute ACVR nin’ in order to reflect 


the degree of robustness desired by the user: 
SC; = SCi + (SPAN, — SC,) x AM!90-ALPHA . (2.5) 


where 


(2.6) 


АСУВ 
AM = ABS ры [оо M 


ACVR, 





The resulting SC, is called the "estimated optimal span" [Ref. 9: p. 10] 
corresponding to I. The set of estimated optimal spans may have an 
unnecessarily high variance, thus they are smoothed using SPAN»; the 


result is the set of optimal spans, SC}. 


Each SC; value is checked using one of the two following logical 
statements in order to verify that the span boundaries fixed by the 
user are not violated: 

ie if SC;SSPAN), then SC;=SPAN, ог; 

2. АҒ 5СІ>5РАМ;, then SC,;=SPAN3. 

Each SC; value is used to estimate a smooth Yi value by interpolating 
between two of the Yis values previously computed. The sign and 


value of F in equation 2.7 forms the basis of the interpolation. 
F=SC,— SPAN, . (2.7) 


If F is negative then equations 2.8 and 2.9, shown below, are used to 


estimate the smooth YU 


-Е 


LL oo o 
SPAN, — SPAN, 


(218) 
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estimated smooth Y; = ((1.0 – Е)х Ур] – |ЕхУџ] . (2.9) 


otherwise equations 2.10 and 2.11, shown below, are used: 


Е 


= SPAN, - SPAN, | (2.10) 


estimated smooth Y, = {(1.0 — F)xYpj = !FxYg! . (2214) 


The final smooth Үү values are obtained by smoothing the estimated 
smooth Y, using SPAN,. This smoothing is done in order to reduce the 
variability of the estimated smooth Үт caused by the variance in the 


input data. 


С. nS E DETAILS---SMOOTHING SUBROUTINE, PRIMARY 


The primary use of the Smoothing Subroutine is to smooth data with 
abscissa values in the interval [1.0, 2.0, ., N]. The secondary 
use of the Smoothing Subroutine is to smooth data with abscissa values 
in the interval (0.0, 1.0]. The Smoothing Subroutine requires that the 


following data be transferred from the Combining Subroutine: 


1. N, the number of points to be smoothed; 

24 the array to be smoothed, in this subroutine this array will be 
referred to as 21, . "e ғы; 
X1, Te Хү, the abscissa that correspond to the 21; 


4. SPAN, the span value; 
a flag, IPER, which indicates whether the cross-validated resi- 
duals are to be computed or not computed; 


6. VSMLSQ, the defined value of zero. 


The size of the neighborhood of points included in the local aver- 
aging is determined by SPAN. Most smoothers will not compute the size 
of the neighborhood and require that the user enter an odd integer 
number indicating the size of the neighborhood. Supersmoother will 
compute the size of the neighborhood, thus allowing the user an infinite 


number of choices, since the value of SPAN, as entered by the user, 
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belongs to the interval (0.0, 1.0]. Since the computation of the neigh- 
borhood results in an integer value, different SPAN value entries may 
result in the same neighborhood size, e.g. if N=100, then SPAN values 
of 0.04 and 0.045 wild both result in a neighborhood size of 4. 
Supersmoother uses two neighborhood sizes, IBW and IT, both integer 
values. The first (IBW*1) smoothed points and the last IBW smoothed 
points of the output Z array are computed differently than the central 
smoothed points. Most smoothers drop IBW points at the beginning and 
at the end of the smoothed Z array, e.g. the Moving Average smoother 
mentioned in Chapter I. IT is the number of points included in the 
local averaging. These IT points are the nearest neighbors of Хү, the 
abscissa corresponding to the  smoothed point being computed. 
Supersmoother wil always compute IT to be an odd integer. IBW, on 
the other hand, sometimes may be odd or even, depending on the value 
of N and SPAN. Since IT is odd, the X, will be the median of the 
neighborhood with L(IT/2), integer division, points to the left and 
right. The following two equations are used in the computation of the 
neighborhoods, IBW and IT: 


IBW = (0.5xSPANxN) ~ 0.5 ; | (2:012) 
ЈЕ = (2:IBW)- 1. ЕТ) 


The first IT values of the X and Z arrays are used to compute the 


base or initial values of У А Z covariance of X and Z, and 


mean’ 
variance of X using equations 2.14 through 2.17: 


ІТ 
AC 
J=1 


n | (2.14) 
Ber can IT ) 
> 
2) 
7 ша, i (22255) 
mean it ` 
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IT 

COV yz = Ух, m X меа) х (2) Ба A) ) (2 16) 
д=1 
ІТ 

УАВх = У(Х - Х....)* . (2.17) 


J=1 


The first (IBW*1) smooth 2] аге computed using the results from equa- 
tions 2.14 through 2.17. The first step in the computation of these 
sheath 21 is to find the slope, A, of the least squares straight line 
through the set of points (X4, Z4), . . ., (Хүт, 2үт) [Ref. 9: p. 5]. 
If VARySVSMLSQ, then A=0.0, otherwise equation 2.18 is used: 


СОМ; 
ҮАК; 





(2.18) 


The second step is the actual computation of the smooth 21] using the 
slope, A, computed with equation 2.18, the results from equations 2.14 


and 2.15 and the following linear equation: 
smooth Z; = Ax(Xy;— Xmean) + Zmean - (2.199 


The cross-validated residual, ACVR,, are computed using the following 
procedure: 
i compute the portion of the neighborhood occupied by the 


smooth point, H, using equation 2.20: 
Н = —= ; (2.20) 


D if VARy>VSMLSQ, then this large degree of variability inherent 


in the raw data must be reflected in H using equation 2.21: 


0,4 us X wan) 
Н-Н--------. (2.210 
VAR, 
3. finally the cross-validated residuals are computed with equation 
222: 
ABS(Z,; — smooth Z,) 
АСУВ,- ----------- (2.22) 


1.0- H 
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Recall that only the first (IBW*1) smoothed Zy values have been 


computed. 


In order to compute the smoothed ZIBWe]: T ZN-IBW:» the 
neighborhood of points has to be moved from one point to the next 
point toward the right. lhis is where the dynamic programming 
computational procedures are useful in cutting the storage space and 
computer time. The results from equations 2.14 through 2.17 are 
updated to reflect the movement of the neighborhood, i.e. the left 
endpoint of the neighborhood will be dropped and the point to the right 
of the right endpoint will enter the neighborhood. Equations 2.23 
through 2.30 are used for each I, where I-(IBW*1), . . ., (N-IBW). 
Equations 2.23 through 2.26 are used to drop a point from the 
neighborhood: 


~ » ШОХ Ыы) = Alt 
-Mmean — ppm : (2.293) 


И Ша) по. (2.24) 
mear IT кез 1 


ШТ niet л). (2025) 


СОМ yz = COV xz = IT EN ) 


IT x (Холт m xs 


ED (2-25) 
VAR, = VAR es 
Equations 2.27 through 2.30 are used to add a point to the 
neighborhood: 
х = ла (2 7) 


IT ' 
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pe WES ence И (2.28) 
mean IT 3 


IT x (X, = Хаа E Zu 


Vxz = COV 
COV 2 xz + T] ; 


(ща и 


" ; IT x (X, n XU 
VARy = ҮАК; ~ гт. Е (2.30) 


The results from equations 2.27 through 2.30 are then used in equa- 
tions 2.18 through 2.22 to compute the smoothed Zy and the cross- 


validated residuals if necessary. 


Тһе Х сап, mean? О ху ome У а values used to compute 
smooth Zy-rRpw are used to compute the smooth 21 values where 
I-(N-IBW*1), . . ., N, i.e. the smooth values at the tail-end of the Z 
array. These mean and variance values are used in equations 2.18 


through 2.22 in the computation of the smooth 21. This procedure 15 
equivalent to the procedure used to compute the smooth 21, where [=1, 
гу (IBW+1). 
D. MATHEMATICAL DETAILS---SMOOTHING SUBROUTINE, 
SECONDARY USE 
The secondary use of the Smoothing Subroutine is to smooth data 
with abscissa values in the interval (0.0, 1.0]. The Smoothing 
Subroutine needs the same data and follows the same steps and equa- 
tions as if the abscissa were in the interval [1.0, 2.0, . . ., NJ]. The 


exceptions are noted in this section. 


When using equations 2.14 through 2.17, the first IBW points and 
the last (IBW+1) points of the X and Z data arrays are used to compute 


the: initial и 


values. Equations 2.14 through 2.17 are then changed in order to 


covariance of X and Z, and the variance of X 


allow these new points to be involved in the computations. Equations 


2.31 through 2.34, shown below, are the result of the change and are 


used in the computation of the initial values of Поа ан аи 
ance of X and Z and variance of X: 
N IBW 
Poor us 
о (231) 


ПІЛІ ТЕГЕН 
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1% 


2) 222. 


J2N-IBW 41 J=1 
Z O i | 
ШІ. ТЕТІ Bw со 
N ву 
COV yz E У) (Х; d X mera) < (Z3 = DE) у; (ху = Ха (2, E Penh ; (2.33) 
J2N-IDW -1 4-і 
қ ” IBW - а 
ЕЕ 2 (Хо= Хама) = 2 (Ар-— Хива) · VERTS 
Jz N-IBW >+] J=! 


The next step in this smoothing procedure is to drop a point from 
the neighborhood. In the previous Section, equations 2.23 through 
2.26 were used for this task, but they cannot be used in this section 
because the input point counter indicates that negative index values are 
computed at the beginning of the computations. The negative indices 
are the result of the last (IBW+1) point being being used in equations 
2.31 through 2.34. Thus to keep the point counter on track let 
K=N+I-IBW-1 and change equations 2.23 through 2.26 as indicated in 
equations 2.35 through 2.38, respectively. Then in order to drop a 
point from the neighborhood, equations 2.35 through 2.38 are used: 


T Ur Эк” | (2.35) 


IT in = ly = 1.0 
EEUU шш PRU (2.36) 


mean ҮТ ug 
: IT x X —- 1.0 - mean = Ta 
COVxz = COVxyz - пок 7 L0 7 XnealX(7 7 тм) ) (2.37) 
IT -1 
| ІТ х Х - 10- леб : 


IT -1 
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Since a point was dropped from the neighborhood the next adjacent 
point on the right boundary of the neighborhood must be entered into 
the neighborhood. Їп the previous Section, equations 2.27 through 
2.30 performed this task, but in order to keep the point counter on 
track these equations must be modified. Therefore, let K=I+IBW and 
the new equations are equations 2.39 through 2.42. Thus to add the 


next point to the neighborhood, these new equations are used: 


ит - xu] - Xx 


У ы 2.39 
Хаа - | (2.39) 
пт _ 0х2...) a 
ће =; (2.40) 
IT | 
1 = ЕЕ) ИСУС 
COVy2z = COVx2 + 1T x (Xk > Xmean) x (Zr — Zmean) : (2.41) 
з= 
ГГх(Хк ЕЕ Xm) 
УАВх = УАВу 2 ——————— . (2.42) 


IT - 1 


The results produced by equations 2.39 through 2.42 are then used 
in equations 2.18 through 2.22 to compute the smooth Ст, where [=1, 
= CIB Wael). 


In order to compute the middle smooth Z ү values, i.e. smooth Zy 
where I-(IBW*2), ., (N-IBW), equations 2.23 through 2.30 and 
equations 2.18 through 2.22 are used as they are, i.e. no changes 


involved. 


The computation of the last (IBW-1) smooth Zp values, i.e. smooth 
21 where I-(N-IBW*1), 
through 2.26 a second time, in order to maintain the point counter on 


track. This change is needed because the first (IBW-1) input points 


., N, involves changing equations 2.23 
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are used to compute these smooth 2т апа the point counter must not 
exceed N, the number of. points to be smoothed. Thus let K=I+IBW-N 
and the result of the change is shown in equations 2.43 through 2.46. 
Equations 2.44 through 2.46 are used to drop a point from the 
neighborhood: 


(IT > Xmean) — Xg + 1.0 


Xmen = WN-l] | (2.43) 


И Linear) Е Zy L 


К (2.44) 
IT - 1 | 
ПОЕТИТЕ РИ БОТ Е) 
ОТ Кого - EX ен een - (2.45) 
ea 
IT x (X + 1.0- Хари)” 
VARy 2 VAR& — ————————— — (2.46) 


ІТ -1 


In order to replace the point dropped from the neighborhood, equa- 
tions 2.27 through 2.30 are used, but in a different form because of 
the same reason that equations 2.23 through 2.26 were changed above. 
Therefore, with K=I-IBW-1 the equations used to add a point to the 
neighborhood are equations 2.47 through 2.50: 


[пт ы хил] А 


р | 2.47 
КО сав IT ? ( ) 
IT - 1 C m 
A. [OT = Ы | (2.48) 
mean IT » 
ІТ X == Ка х а V EO 
COV yz = COVyz zt M eR == ; (2.49) 
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х (Хк B X sen |: 


УАК; = VAR, zm IT — 1 


(2.50) 


The results produced by equations 2.47 through 2.50 are then used 
in equations 2.18 through 2.22 in order to compute the smooth 21 мћеге 
I=(N-IBW+1), . . ., М. The disadvantage of using abscissa values 
from the interval (0.0, 1.0] versus abscissa from the interval [1.0, 
2.0, . . ., N] is that a slight degree of distortion is produced at the 
ends of the smoothed Z array. The distortion could be caused by the 
1.0 adjustment factor in equations 2.35, 2.36, 2.43, and 2.44. 


E. SELECTION OF SPAN 


The span value is the parameter that controls the smoothing of a 
data set. There exist no set procedures for selecting a span value. 
Each data analyst has his/her own method of selecting the span value. 
The analyst's experience with smoothers determines how the span is 
selected. Selection of the span value is basically a subjective process, 
where the analyst uses a span value which gives adequate and useful 
results. The user of the advanced smoothers should develop a consis- 
tent, span selection process. A common procedure used by some expert 
smoothers starts by looking at a scatterplot of the raw data. Then the 
analyst looks for periodicity and cyclic changes present in the data. 
This information is then used to estimate the span value to be used in 
the smoothing. For example, if a data set displays a cycle of about 24 
points, then the span to use should be about 24/N, where N is number 
of points to be smoothed. This span value is a good estimate because 
the raw data is permitted to determine the shape of the smooth results. 
This procedure is used in Chapter V of this thesis in the smoothing of 


a large set of sea-surface temperatures. 


Supersmoother is unique among smoothing algorithms in that three 
span values, i.e. SPAN,, SPAN», and SPAN}, may be entered by the 
user.  Supersmoother will then select an optimal span value within the 
range of the smallest span value and the largest span value by using 
the method of cross-validation which was explained earlier in this 


Chapter. This option within Supersmoother lets the user be very 
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flexible in the selection of the span values to use. But the user must 
be careful about which span values to use with Supersmoother. It is 
best to first try span values of 0.05, 0.2, and 0.5, as recommended by 
Friedman and Stuetzle [Ref. 9: p. 9]. This range of span values gives 
Supersmoother good coverage of the data. After viewing the results 
‘produced by Supersmoother, the user can adjust the span values in 
order to get the desired smooth effect. When adjusting, the user must 
bear in mind the bias/variance trade-off discuss earlier in this chapter. 
The trade-off being that if the span value is increased, then result is a 
smoother looking curve, while the reverse occurs when decreasing the 


span value. 


No matter what rule is followed to determine the span values used in 
Supersmoother, the final smooth results accepted are based on subjec- 


tive needs, applications, and preferences. 
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III. TECHNICAL DESCRIPTION OF SPLIT LINEAR FIT ALGORITHM 
A. OVERVIEW 


The Split Linear Fit smoothing algorithm was developed at Stanford 
University by John A. McDonald and Art B. Owen. [Ref. 10]. The 
Split Linear Fit smoother produces piece-wise smooth curves and thus 
will depict discontinuities present in the input data [Ref. 10: p. I1]. 
Most smoothers tend to distort discontinuities because the weighted 
averaging technique used to compute a smoothed point requires a 
continuous underlying function. The Split Linear Fit smoother will not 
distort the smooth curve at discontinuous points and does a very good 
job of detecting sharp slopes in the input data. This is the reason the 
Split Linear Fit algorithm is sometimes classified as an edge-detecting 
smoother [Ref. 10: p. 2]. 


The Split Linear Fit smoother is similar to Friedman and Stuetzle's 
Supersmoother in several ways: 

i every input point receives a respective smooth point; 

2 the user can enter more than one neighborhood size; in this 
algorithm the neighborhood sizes are called window sizes, where 
window of size K is defined as "a set of K successive point" 
[Ref. 10: p. 2], (window size and span are equivalent terms); 

3. the window is shifted to the right by dropping the left 
endpoint and then adding the point adjacent to the right 
endpoint; 

4. the method of least squares is used to estimate a straight line 
through the points within the shifting window; 

9x the Split Linear Fit smoother is scale independent. 

A major difference between the Split Linear Fit smoother and 
Supersmoother is the method used to combine the linear fitted values. 
Another difference is that the Split linear Fit smoother does only robust 


smoothing. 
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The objective of the Split Linear Fit smoother is to produce a piece- 
wise smoothed curve with minimal discontinuous features [Ref. 10: p. 
2]. Figure 3.1 shows the Split Linear Fit smoothing algorithm as 
composed of three subroutines: 

1. the Regression Subroutine; 

2. the Weighting Subroutine; 


З. the Combining Subroutine. 


Input 


Regression 
Subroutine 


Regression 
Subroutine 


Weighting 
Subroutine 


Weighting 
Subroutine 


Combining 
Subroutine 


Combining 
Subroutine 


Primary Output Final 


Output 


Figure 3.1 | Data Flow in Split Linear Fit Smoother. 


Figure 3.1 also shows that the Split Linear Fit smoother uses the itera- 
tive process once on its output. This is done in order to decrease the 
variance in the first set of output, since, as mentioned before, the 
Split Linear Fit smoother only does robust smoothing. If the first set 
of output were to be plotted, the curve would appear very jagged. 
Passing the first set of output through the Split Linear Fit algorithm, 


decreases the robustness and variability of the final output. 
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The number of window sizes entered by the user dictates the 
number of times that the input data is passed through the Regression 
Subroutine to produce a family of linear fitted values and residual 
values associated with each I, I-1, . . ., М, where N is the number of 
points to be smoothed, see Figure 3.2. Each family of linear fits may 
be viewed as a pseudo-distribution of linearly fitted estimated values of 


an input point value. 


Select window size 


For each input point, compute 
linear fitted values and 
mean squared residual values 


Last 
window 
size 
? 


Yes 


N families of 
linear fitted values and 
mean squared residual values 
to Weighting Subroutine 





Figure 3.2 Regression Subroutine in Generalized Form. 
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The Weighting Subroutine receives the N families of linear fits and 
mean squared residual values from the Regression Subroutine and finds 
the minimum mean squared residual value within each family of mean 
squared residual values. Not all the mean squared residual values 
qualify as candidates for the minimum mean squared residual values. 
An acceptable mean squared residual value is one that does not exceed 
an established cutoff value. The cutoff value used in the Split Linear 
Fit algorithm is the value -1.0х1030. This value was selected because 
it provided a better smooth curve at discontinuities inherent in the 
raw, input data than other cutoff values [Ref. 10: p. 3]. The minimum 
mean squared residual value is used as a base to compute a weight 
corresponding to each of the acceptable mean squared residual values 
within the family, see Figure 3.3. These weights are used by the 
Combining Subroutine in computing the smooth point values. The 
weights "depend on a measure of the quality of the corresponding linear 
fits" [Ref. 10: р. 2]. Quality meaning that the smaller the mean 
squared residual value the higher the weight assigned to the corre- 
sponding fitted value. The weight assigned to a fitted value is a func- 


tion of the following: 


ШЕ the corresponding mean squared residual; 
2. the minimum mean squared residual, and; 
3. the average of the acceptable mean squared residuals within the 


associated window. 
This weighting procedure is used in order to smoothly integrate discon- 
tinuities in the input data with the other smooth points. This proce- 


dure is the edge-detector and is the cause of the robust smoothing. 


The smooth point value at I is a weighted average of the linear fits 
in the family of linear fits corresponding to I. The Combining 
Subroutine combines the weights produced by the Weighting Subroutine 
and the fitted values produced by the Regression Subroutine associated 


with I and computes the respective smooth point value, see Figure 3.4. 


As mentioned before the first set of smooth point values produced 


by the Combining Subroutine is itself passed through the Split Linear 
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Inout from 
Regression 
Subroutine 


For Ith family find 
minimum mean squared 
residual value 


Compute weight 
corresponding to 
each mean squared 
residual value in 


Ith family 


N families of 
weights to 
Combining 
Subroutine 





Figure 3.3 Weighting Subroutine in Generalized Form. 


Fit algorithm again in order to reduce the variability. The second set 


of smooth point values is the output generated to the user, see Figure 
254 |5; 


B. MATHEMATICAL DETAILS---REGRESSION SUBROUTINE 


The Regression Subroutine requires the following user input: 
1. МА, the number of points to be smoothed; 
2 т ЗЕ: зе the point values to be smoothed (in chronolo- 


gical order); 
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Input from 
Weighting 
Subroutine 












Select Ith family of 
linear fitted values 
and weights 





Compute Ith 
smooth point value 


No 
Output 


Figure 3.4 Combining Subroutine in Generalized Form. 


3. Xi, . . ., Xy, the abscissa corresponding to the Y values (in 
ascending order since the Yi are in chronological order); 
NTRYS, the number of window sizes to be used; 
WNSZ4, E Es WNSZw'TRYS.: the values of the window sizes; 
MNWNSZ, the minimum window size permitted by the user. 
The minimum window size, MNWNSZ, is the lower bound set on the 
window size. The value of the lower bound should be at most one-half 
the value of the smallest window that will be used in the smoothing. If 
MNWNSZ is any larger then some smooth points will be dropped from the 


ends of the output array, or a plot of the smooth point values will show 


distortion at the ends. 


45 


Figure 3.2 shows the Regression Subroutine using a procedure 
resembling the iterative process, but each pass of the Үү values 
through the Regression Subroutine uses a different window size, and 
some variables are reset to zero. The purpose of the Regression 
Subroutine is to compute a family of fitted values and a family of mean 
squared residual values corresponding to each I, I- 1, . . ., N. The 
Regression Subroutine is shown in more detail in Figure 3.5. The 
Regression Subroutine can be divided into three parts: 

1. definition of zero and computation of sum of first (MNWNSZ-1) 
values and computation of fitted values for 1-1, 
(MNWNSZ-1); 

25 shifting of window and computation of fitted values and mean 

„› (N-MNWNSZS*IDE 
3. | computation of fitted values for I-(N-MNWNSZ*2), ДЫ 


У 


squared residual values for I-ZMNWNSZ, 


The variable EPS is used to define zero for computational purposes. 
The interquartile range of the abscissa array is used in the computation 


of EPS, as shown by equations 3.1 through 3.3: 


ПЕ (3.1) 
4 

ЛЬ сай) 

EPS = XR- ïn. (3.3) 


If EPSSO.0 and JR«N, then EPS is recomputed using the following three 
rules: 

1. if JR<N, then JR is increased by a value of one; 

2. if JL>1, then JL is decreased by a value of one; 

Jj. EPS is recomputed using the new values of JR and JL and 

equation 3.3. 

EPS will be equal to zero only if N<3 and if the computer allows index 
values equal to zero, otherwise a computer error will result. If this 
Situation occurs then items 1 and 3 from above will apply. Since the Хү 


values are іп ascending order, EPS will have a value greater than zero 
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after the first recomputation of JR. After EPS has been defined as 
being greater than zero, it is adjusted using equation 3.4 in order to 


define zero for computational purposes: 


EPS = (EPS x 1.0107")? | | (3.4) 


In order to fit a linear model on the window endpoints and central 
point, slope, and y-intercept values of the model must first be 
computed. The parameter MNWNSZ dictates which input point values 
wil be used to compute the initial slope and the corresponding 
y-intercept value. The first MNWNSZ values of the input data are used 
to compute these necessary values. The first (MNWNSZ-1) values of 
the input data are used in equations 3.5 through 3.10 to compute the 
basic sum values to be used later and increment a counter which ое. 


track of the input points: 


MNWNSZ-1 Ё 
І= 1 
MNWNSZ-1 
ШМ y» Y; (3.6) 
11 
DEGUNTER - MNWNSZ - 1 ; (3.7) 
MN WNSZ- 1 
SUMmxsq = у) Das ; (3-6) 
1= 1 
MNWNSZ-1 
У ОМ, о = Үү ) (329) 


1= 1 
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Figure 3.5 Data Flow in the Regression Subroutine. 
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MNWNSZ-1 


SUMxy= У! (хү) . (3.10) 


1= 1 


For each I-1, ., (MNWNSZ-1), the right endpoint fitted value and 
the mean squared residual value are set to the value of -1.0х1099. 
This is done so that the Weighting Subroutine will assign a zero weight 
to the left endpoint fitted values and the mean squared residual values 
corresponding to these I. Therefore, the smooth values corresponding 
to these I values are computed using a smaller window [Ref. 10: p. 3]. 
This procedure does not use the window concept that is mentioned 


below. This is done to avoid computer errors since most of the points 


in the window corresponding to these I will have negative index values. 


Figure 3.5 shows the iterative process that is used to compute the 
family of fitted values and the family of mean squared residual values 
corresponding to I, I=-MNWNSZ, ., (М-ММММ5 0+1). The first step 
determines the window central point and the endpoints that correspond 
to I. For ease of understanding, let K be the number of successive 
points in the window. If K is odd then the central point is equivalent 
to the median of the window and has L(K/2) neighboring points to the 
left and right of it. If K is an even number, then the central point is 
wil have [(K/2)-1] neighboring points to the left and (K/2) neigh- 
boring points to the right, thus the central point is the point to left of 
the window median. The index of the right endpoint will always be 
equal to I. If K is odd, the index of the central point will be equal to 
[I- L(K/2)], and the index of the left endpoint will be equal to 
(I-K*1). If the value of K is even, the index of the central point will 
be equal to (I-(K/2)], and the index of the left endpoint will be equal 
to (I-K*1). Point values that have corresponding index values that are 


negative or zero are not included in the linear fit. 


The next step adds the ith point to the window and uses the 
method of least squares to estimate the straight line through the points 
in the window. The procedure for adding the th point to the neigh- 
borhood adds the values produced by equations 3.5 through 3.10 to the 


Хү and the ЕТ values using equations 3.11 through 3.16: 
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Us ee (3719) 


SUM, + У, ; (3.12) 


SUMy = 

KOUNTER = KOUNTER +1 ; (3.13) 
SUMx5q = SUMxsq + ХР; (3.14) 
SUMysq = SUMysq + Yr ; (3.15) 
SUMyy = SUMyy + (XixY)) . (3.16) 


Next the mean of the sum values computed by equations 3.11 through 
3.16 is computed using the value of KOUNTER as the denominator in 
equations 3.17 through 3.21: 


MEAN, 2 ОМ _ | 
* AW KOUNTER 3 (3.199 
SUM, 
N Му = —————— : 3.16 
MEPS’ СІНЕН ( 
МА а = с=ш=; 3.19 
MEANxsa = KOUNTER ү ) 
5 Муза 
МЕАГузо = KOUNTER | (3.20) 
MEAN ЕАУ 3.28 
ЕИО КОКЕ a 
The variance of the abscissa is derived by equation 3.22: 
XVAR = MEANysq — MEAN} . (3.22) 


The method of least squares is used to compute the slope and the 
y-intercept of the straight line fitted to the points in the window. The 
results produced by equations 3.17 through 3.18 are used to compute 
the coefficients of the straight line that is fitted to the points within 


o0 





the window. If XVAR<0.0, then the slope of the straight line, SLOPE, 
is zero, i.e. SLOPE = 0.0, otherwise the value of SLOPE is computed 
with equation 3.23: 


MEANyy – (MEAN, x MEAN 
SLOPE = — ore | (8:73) 


The y-intercept of the straight line is computed using equation 3.24: 


INTER » MEAN, - (SLOPE » MEANy) . (3.24) 


The mean squared residual value about the fitted line is computed with 


equations 3.25 through 3.28: 


МЕАМзо= А+В+С ; (5) 
where 

A- MEANysq  - (2xINTERxMEANy,) - (2xSLOPExMEANyy) ; (3.26) 

В = INTER? « (2xINTERxSLOPExMEANy) ; (3.27) 

C - MEANysqxSLOPE. . (3.28) 


The window central point and endpoints are fitted to a linear model 
using the slope and the y-intercept value computed above to produce 
the fitted value FIT wp, where I= current ith value, W= current window 
size, and P= left endpoint, central point, or right endpoint. The mean 
squared residual value, MSQRiwp, is computed using the computed local 
linear fit coefficients, the Xj and Y, values, and the counter value in 


equations 3.29 through 3.31: 
FITiwp - INTER ^ (SLOPExXj) ; (3.29) 


RES - Yi - РІТур ; (3.30) 


51 


(KOUNTER xMEANgsq ) ~ RES? 


MSQRywp = KOUNTER - 1 


(3:31) 
After FIT}wp and MSQRiyp have been computed for the ith 
window's central point and endpoints, the window must be shifted to 
the right. The Supersmoother algorithm uses the same procedure as 
the Split Linear Fit, i.e. dropping the left endpoint from the sum 
values of equations 3.11 through 3.16 and then adding the entering 
point to these same equations. Let IL be the index of the left 


endpoint, then this point is dropped using equations 3.32 through 3.37: 


SUM, = SUMx = Xn 3 (39990 
SUM = SU a. (3.33) 
KOUNTER = KOUNTER р. | | (3.34) 
ИМ уска = УМ = Xp. | (3:22) 
о — е (3.36) 
SUMyy = ОУ) (3.31) 


Next, I is incremented by one and, if Is(N-MNWNSZ+1), equations 3.11 
through 3.16 are used to enter the new jth point into window. 
Equations 3.17 through 3.37 are then repeated using the new values. 


This procedure is continued until I>(N-MNWNSZ?1). 


When I>(N-MNWNSZ+1), the left endpoint ЕГГуур and MSQRiwp 
values corresponding to the values of I are set equal to -1.0х1030, SO 
that these fitted values are assigned no weight in the Weighting 
Subroutine. This procedure was used for I= 1, „› (ММ з ТОҒАН 


the beginning of the Regression Subroutine. 


If the user entered more than one window size, then the input data 


is passed through the Regression Subroutine with the next window size. 
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The sums and counter of equations 3.5 through 3.10 are initialized to 
zero before repeating the Regression Subroutine. After the last window 
size has been used, each window will have contributed a total of six 
values to each I, i.e. three linear fitted values and three mean squared 


residual values. 
C. MATHEMATICAL DETAILS---WEIGHTING SUBROUTINE 


The objective of the Weighting Subroutine is to compute a weight 
which is indicative of the degree of goodness of fit of each linear fitted 
value, FIT Twp» with respect to a line with slope equal to one. Figure 
3.3 shows the procedure followed by the Weighting Subroutine. As 
noted in the figure, each family of mean squared residual values is 
used one set at a time. The following data is transferred from the 
linear Regression Subroutine: 

1.  N,the number of points to be smoothed; 

D. NTRYS,the number of windows used in the smoothing; 

3. N families of mean squared residual values, MSQRiwp:; 

4. N families of fitted values, FITiwp. 

The Weighting Subroutine is executed once for each family of mean 
squared residual values. The Regression Subroutine produced a family 
of (3xNTRYS) mean squared residual values corresponding to each I. 
For computational feasibility a lower bound of -1.0х1099 і set on the 
values of mean squared residual, i.e. the MSQRiwp. Each MSQRivp 
value is compared against -1.0x10°9, and the MSQRywp values less than 
or equal to -1.0х1030 аге marked as unacceptable. These are not 
considered in the search for the minimum MSQRiwp within the jth family 
MSQRywp- The minimum MSQRiwp corresponding to I is found by doing 
a comparison between the acceptable MSQRiwp values in the тр family 
of MSQR wp. The expressions listed below are used by the Weighting 
Subroutine on each family of MSQRywp: 

ПЕ MIN is the minimum MSQRiwp in th family; 

2. LAMBDA is the sum of MSQRywp greater than -1.0х1099, 

d. LAMBDA is divided by the number of MSQRīwp greater than 

-1.0х1099. 
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4. LAMBDA is then reduced by the value of MIN. 
If the value of LAMBDA is less than or equal to zero, then LAMBDA is 
not modified, otherwise LAMBDA is recomputed using equation 3.38: 
1.0 


_ 18 3.38 
LAMBA = TAMBA (5:55 


If the value of MIN is positive, MIN is not changed, otherwise it is 


made a positive value using equation 3.39: 
MIN = 1x10" . (3.39) 


The last step in the Weighting Subroutine is to compute a family of 
weights which indicate the goodness of fit of the linear fitted values, 
FiTjwp, using the corresponding family of MSQRiwp values. The 
MSQRiwp values which are less than -1.0х1099 cause a weight of zero 
to be attached to the corresponding fitted value, FIT Iwp:. The reason 
for this occurring is that these values are considered unacceptable 
based on the established cutoff value discussed in Section A of this 
chapter. If the MSQRiwp value satisfies the cutoff rule, then a weight 
will be computed indicating the goodness of fit of the corresponding 
FITiwp. The weight is a function of the quality of the corresponding 
MSQRiwp value. TEMP indicates the degree of quality and is computed 


using equation 3.40: 
TEMP = LAMBA ВОТ ПЕ (3.40) 


Recall that the smaller the value of MSQRrwp, the better the value 
of FIT wp. In other words, small values of TEMP indicate a good 
FIT Twp value, therefore, thesa fit values receive high weights. Three 
conditions are used in assigning a weight that to each acceptable 
МОК јр: 

ШЕ if ТЕМР<0.0, then WIiwp rO: 

e if 0.О<ТЕМР<1.0, then the weight is computed using equation 

Sade 
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WTiwp = (1.0 = TEMP)? ; (3.41) 


3. if TEMP21.0, then WT yp = 0.0. 
According to the Split Linear Fit developer, equation 3.41 results in 
"smooth transitions between zero weights and small non-zero weights" 
while other functions tend not to have the desired effect [Ref. 10: p. 
3]. Recall that the Weighting Subroutine is repeated for each family of 
mean squared residual values and that the output is a family of weights 


corresponding to each I. 
D. MATHEMATICAL DETAILS---COMBINING SUBROUTINE 


The objective of the Combining Subroutine is to compute the smooth 
point values. The following data is transferred from the Weighting 
Subroutine: 

1.  N, the number of points to be smoothed; 

à NTRYS, the number of bandwidths used in the smoothing; 

За N families of fitted values, E трі | 

4 N families of weights, WTiwp- 


Before using a ЕТ тур value in the following computations, it is 
compared to the value -1.0х1039. В р values less than or equal to 
-1.0х1030 are marked as unacceptable and not used in the computations 
of the corresponding smooth value. Using each family of WT iwp and 
FITiwp, the Combining Subroutine computes a weighted average of the 
linear fitted values, FITīwp. The first step in computing the th 
smooth point is to use the corresponding family of FIT wp and ҮТ түүр 


values in equations 3.42 and 3.43: 


RSCM = SO SOFIT we . for [= To NS қ A (3.42) 
М Р 

МОМ = 3,9, Тур, ог 1=1,2,...№ ; (3.43) 
УР 


If WSUM21.0x10 19, then the th smooth point is produced by equation 
3.44, otherwise SMOOTH, equals -1.0х1030. 
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RSUM 





(3.44) 


Next, I is incremented by one, and the Combining Subroutine is 
repeated using the new value of I and the corresponding families of 
ELIP and WT wp values. This procedure continues until I>N. If 
these smoothed point values were plotted, the plot would show too many 
peaks and would appear very jagged. In order to alleviate this 
problem, these smoothed point values are used as input data to the 
Regression Subroutine, and the Split Linear Fit algorithm is executed 
one more time. The output of the second pass does not have as much 
variability as the output from the first pass and is thus more useable 


for data reduction. 
E. SELECTION OF WINDOW SIZE 


The section on selection of span value in the previous chapter 
applies in this chapter to a great degree. Window size and span value 
are equivalent terms, and both affect the smoothing to a great degree. 
Since both the Supersmoother and the Split Linear Fit use local linear 
regression, the relationship between a window size and the degree of 
smoothing can be explained by equation 1.4. The effect on the residual 
values caused by varying the window size in equation 1.4 must be kept 
in mind when selecting a window size, i.e. remember the following: 

qu large window sizes produce a smooth plot; 

2. small window sizes produce a not so smooth plot. 

What equation 1.4 is illustrating is that the degree of smoothness is the 
result of a tradeoff between bias and variance in the resulting smooth 
plot since it is an estimation of a function in the presence of additive 
errors [Ref. 10: p. 1]. А satisfactory trade-off between bias and 
variance is difficult to obtain. Better decisions can be made by looking 
at a plot of the smooth output. Therefore, the user of a smoothing 
program should plot the smooth output and then decide if the results 
satisfy his/her needs and desires, otherwise the smoothing program is 
run again with a different window size. Some people want and need 
very 'smooth' and highly biased results while others want results on the 


other extreme, i.e low bias and high variance. 
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The above paragraph explains in general the effects produced by 
using one window size, but the Split Linear Fit smoother can accept 
more than one window size. By using more than one window size the 
desired smooth output may be found in less time than if a single window 
size had been used. The reason for this speed is that the Split Linear 
Fit algorithm has more information about the shape of the raw data, 
than when only one window size is used, i.e. a pseudo-distribution of 
fitted values about each raw point is produced. Then the smoothed 
point is computed using this additional information, i.e. a weighted- 
average of the fitted values. But the user needs to apply the basic 
relationship between window size and degree of smoothness stated in the 
above paragraph in selecting a set of window sizes, i.e. a set of 'large' 


windows will produce a smoother effect than a set of ‘small’ windows. 


According to McDonald and Owen it is best to use a set of three to 
five consecutive odd window sizes [Ref. 10: pp. 2-4]. A mixture of 
small and large window sizes will result in centrally smooth point values 
with a slight degree of variability. In order to be able to accurately 
trace the curvature of the input data, it is best to do the following: 


1. roughly measure the periodicity of the input data; 


2. use this value as one of the window sizes to be used in the 
smoothing; 

D. select the other window sizes with respect to the value of the 
periodicity. 


For example, if the periodicity of the input data is estimated to be 27, 
then 27 is used as an input window size and the other window sizes 
may be 23, 25, 29, and 31. Or the periodicity value may be either the 
smallest window size or the largest window size while the other window 


Sizes are selected with respect to the periodicity. 


The other factor that has great influence on the smooth output 
produced by the Split Linear Fit is the minimum window size, MNWNSZ. 
If this value is too large then the smooth output will not be what is 
expected. It is best to keep the value of MNWNSZ at no more than 


one-half the value of the smallest window. This subject wil be 


о 


discussed in more detail in the evaluation of the Split Linear Fit 


smoother. 
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IV. EVALUATION OF THE ADVANCED SMOOTHING ALGORITHMS 


Sm a у шш ë отит, 


A. GENERAL 


As stated in Chapter I, a smoothing algorithm may be compared to a 
low-pass filter which is designed to do the following: 

1. filter out "noise" from a data set and; 

2. estimate the underlying functional relationship present in the 

given data set. 

Before proposing a 'good and efficient! smoothing algorithm to an indi- 
vidual, the user must be shown that the 'good and efficient! smoothing 
algorithm is robust. [In other words, it is necessary to illustrate that 
the smoothing algorithm performs well with most data sets, whether the 


underlying function is either of the following: 


ili, simple like a linear function or a simple trigonometric function, 
ог; 
Z. complex and is very difficult to define mathematically. 


In this chapter, the input data sets used with the Supersmoother 
and the Split Linear Fit smoothing algorithms are generated from simple 
known functions with Normal (0,1) "noise" added. The GRAFSTAT 
[Ref. 12] functions used to generate the pseudo-random Normal (0,1) 
deviates are given in Section 1 of Appendix C. The output produced 
by these algorithms is evaluated in order to do the following: 

l. observe how well the Normal (0,1) "noise" is filtered by the 

smoother, and; 


2 determine how well the true function is extracted and depicted. 


In this chapter the input data sets have a constant variance of 1. 
In the next chapter the input data set dose not necessarily have a 
constant variance, because it is real, and the unknown underlying 


function is probably too complex to define succinctly. 
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B. METHODOLOGY 


There is no established procedure to follow in testing the efficiency 
and effectiveness of а smoothing technique/algorithm. Since the 
adequacy or usefulness of he smooth output is largely based on the 
user's subjective judgement of the shape of the smooth curve, i.e. a 
plot of the smooth point values, the most effective method is to compare 
the output produced by the new algorithm to the output produced by 
previously validated, widely used, and well liked smoothing algorithms, 
e.g. LOWESS [Ref. 3: pp. 94-104]. The following procedure is used to 
evaluate the Supersmoother and the Split Linear Fit smoothing 
algorithms: 

ТЕ explain and display the data set to be smoothed; 

2c display and explain the smooth results produced by the base- 

line smoothing  techniques/algorithms, i.e. Least Squares 
Regression, Equal-Weight Moving Average,  Cosine-Weighted 
Moving Average, and LOWESS; 

3: display and examine the smooth results produced by the 

advanced smoothers; 

4. compare these results to the previously discussed results from 


2 above. 


The GRAFSTAT graphics package [Ref. 12] was used to generate all 
of the graphs in this thesis. GRAFSTAT was also used to do the Least 
Squares Regression and the Equal-Weight Moving Average smoothing. 


The Method of Least Squares tries to standardize the curves that 
can be fitted to a data set. The measure of performance that is used 
with this global fitting technique is the sum of squared residuals. The 
Method of Least Squares produces a smooth curve which closely approx- 
imates the given set of data points and which minimizes the sum of 
squared residuals attainable with the chosen global curve. For more 
explicit details see Spiegel [Ref. 13: pp. 258-305]. GRAFSTAT lets the 
user select the type of curve that should be fitted to the given data 
set. The following listed curve fits which use the Method of Least 
Squares were used in this thesis and are available in the GRAFSTAT 


graphics package: 
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1 linear curve fit; 

2. quadratic curve fit; 

3 third degree polynomial curve fit, and; 

4. Spline fit. 
Least Squares Regression with linear fit is a technique of finding the 
linear equation which fits a data set and minimizes the sum of squared 
residuals. Least Squares Regression with quadratic fit does the same, 
but the data is fitted to an second degree polynomial equation, i.e. Y = 
АХ? + BX + C, where A, B, and C are the estimated coefficients, X is 
the abscissa corresponding to the data being smoothed, and Y is an 
estimate of the data being smoothed. Least Squares Regression with 
third degree polynomial curve fit is also basically the same as previous 
two techniques, but the equation being fitted to the given data has the 
form of Y - AX?'« BX? + CX + D, where A, B, C, and D are the esti- 
mated coefficients and X and Y are the same as for the quadratic fit. 
For more details about Least Squares Regression with either linear fit, 
quadratic fit, or third degree polynomial fit see Spiegel [Ref. 13: pp. 
258-305]. All of these techniques use global curve fitting, i.e. the 
curve is fitted to the given data as an entity. The Spline fit on the 
other hand uses local curve fitting in order to produce the smoothest 
possible curve with the sum of squared residuals value less than or 
equal to a parameter entered by the user [Ref. 12]. The Spline curve 
fitting technique uses the Least Squares Method with third degree poly- 
nomial within a predetermined neighborhood of the given data. The 
neighborhood size is predetermined by the developers of this 
GRAFSTAT function. The second derivative of the defined cubic equa- 
tion is computed and evaluated using the median of the neighborhood of 
point values. The neighborhood is then shifted to the next point and 
the procedure is repeated. The sum of squared residuals is computed 
and compared to the maximum sum of squared residuals value that the 
user requires. If this value exceeds the users constraint then the 
entire procedure is repeated. In other words, the Spline curve fitting 
technique is a constrained linear programming problem, where the 


constraint is the user's maximum sum of squared residuals value 
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desired, and the objective function is a cubic equation [Ref. 16: pp. 
77-87]. The user of these curve fitting technique should first look at 
a scatterplot of the raw data and then decide which one to use. 
Therefore, if a scatterplot of a data set looks linear, the user should 
attempt to fit a linear curve to the data, otherwise, one of the other 


curve fitting techniques should be used. 


The Equal-Weight Moving Average smoother was briefly discussed in 
Chapter I during the discussion of equations 1.3 and 1.4. The 
Equal-Weight Moving Average GRAFSTAT functions that were used to 
generate the smooth point values are shown in Appendix C. The 
following equation defines the smooth points produced by the 


Equal-Weight Moving Average smoother: 
s(X eun = Е x] . [у= 1,2. (N- K), (4.1) 


where N is the number of points to be smoothed and K is the neighbor- 
hood size, i.e. the number of points involved in the averaging. Both 
N and K must be positive, non-zero integers, with K being odd. For 
an expansion of the Equal-Weight Moving Average smoothing theory, see 
Anscombe [Ref. 14: pp. 153-159]. 


The Cosine Weighted Moving Average smoother is an extension of 
the Equal-Weight Moving Average smoother. Instead of using the 
inverse of K as the weight for each Y value within the neighborhood, a 
cosine related weight is computed for each of the Y values within the 
neighborhood of size K. (The APL functions used to generate these 
values appear in Appendix C.) These cosine weights are a function of 
the Y values' location within the shifting window/neighborhood of size 
K. The expression defining the smoothed output of the 


Cosine-Weighted Moving Average smoother is: 


K 
Хо ка) - 5% (тууа) ; ] = 1.2: AER (4.2) 
J=1 


2 


where 
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Anscombe [Ref. 14: p. 450] characterizes this smoother as a "better- 
quality approximation" than other Moving Average smoothers, because 
"not only does the integrand, but also its first derivative vanish at 


both ends of the range of integration" [Ref. 14: pp. 156-157]. 


The last smoother used as a base against which the advanced 
smoothers were tested is the Locally Weighted Regression Scatter Plot 
Smoother, commonly called LOWESS [Ref. 3: pp. 94-104]. LOWESS uses 
the smoothing technique referred to as local regression, i.e. the 
Method of Least Squares is used on the input points within a user 
given neighborhood. Only one neighborhood Size is used by LOWESS 
per run of the program. The program requires that the user not enter 
the neighborhood size to be used, but a parameter called F, which is a 
ratio of the neighborhood size to the number of points to be smoothed. 
The user has the option of fitting either a linear or a quadratic func- 
tion to the point values within the neighborhood. In addition, the user 
has the option of using robust or non-robust smoothing. Robust 
smoothing has more variability than non-robust smoothing, because 
outliers are emphasized. Each input point within the neighborhood 
receives a weight which is a function of the point's location with 
respect to the median of the neighborhood. These weighted point 
values are then used to define a fitted curve within the neighborhood 
of input point values. The coefficients of the defined curve and the 
median of the neighborhood are used to compute the smoothed point 
value corresponding to the median of the neighborhood. The neighbor- 
hood size is shifted from one point to the next until each input point 


has a corresponding smoothed point value. 


Each smoother being used in this thesis requires that a neighbor- 
hood be indicated by the user, but each of these smoothers calls the 
neighborhood size by a different name as discussed in this section. 
The term 'neighborhood size', i.e. the number of point values involved 
in the averaging, can be used by any of the smoothers being discussed 
in this thesis. The Moving Average smoothers use the variable M to 


indicate the neighborhood size. The LOWESS smoother uses the 
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variable F, as discussed the above paragraph. Supersmoother uses a 
value called SPAN which is equivalent to the F value of LOWESS. The 
Split Linear Fit smoother uses the term ‘window size' which is equiva- 


lent to the M value of the Moving Average smoothers. 
C. TESTING AND RESULTS----LINEAR UNDERLYING FUNCTION 


The first test posed on the Supersmoother and the Split Linear Fit 
algorithms was to detect linear trends in a data set which does have a 
linear trend. Figure 4.1 shows Test Set One which consists of 200 data 


points produced from the following equation: 
Y = X + Normal (0,1) noise, 0<X<200 . (4.3) 


The values produced by this function are in tabular form in Appendix 
D. Figure 4.2 shows the results from doing a linear regression on Test 
Set One. It is obvious that the linear regression curve and the true 
linear curve do not coincide. A Confidence Interval Test on the coeffi- 
cients produced by the linear regression reveals that the Y-intercept 
coefficient, 0.0023573 is not significantly different from zero with a 
Confidence Level greater than 0.8. The slope coefficient has a 
Confidence Level less than 0.001 that it is not significantly different 
from zero. ‘Therefore, the linear regression curve can be reduced to 

Y = 1.0104X which has a standard deviation of 0.031 which includes the 


true linear relationship, Y = X. 


The LOWESS smoothing results are shown in Figure 4.3. Since Test 
Set One appears to be linear, the linear option of LOWESS is used. 
There is little visible difference between the results produced by using 
the robust option and the results produced by using the non-robust 
option, i.e. compare the left-hand smooth plots with the right-hand 
smooth plots of Figure 4.3. This is not surprising since there are no 
outliers in this artificial data. The graphs in Figure 4.3 show that as 
the F value is increased, the curve produced gets smoother, i.e. as the 
neighborhood size increases the curve gets smoother because the bias 


increases and the variance decreases. Al F values greater than or 
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TEST SET ONE WITH N(0,1) NOISE TEST SET ONE WITHOUT N(O,1) NOISE 





Figure 4.1 Test Set One. 


TEST SET ONE: LINEAR REGRESSION 


5000 UNE: Y - 0.0023573 * 1.0104X 
OOTTED: Y e X 





Figure 4.2 Test Set One: Linear Regression. 


equal to 0.5 returned the same smooth point values and thus have the 


same plot. 
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Figure 4.3 Test Set One: LOWESS Smoothing. 
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Therefore, an individual who has little knowledge of the LOWESS 
theory should get quick, reasonable results if the F value used is 
between 0.33 and 0.66 as suggested by Chambers, etal [Ref. 3: p. 98]. 
The LOWESS smoothing program may have to be run several times 


before a straight line is produced. 


In Figure 4.4 are shown the smoothing results produced by the 
Supersmoother algorithm. The graphs on the left-hand side display the 
curves produced when only one span value is used. The graphs on the 
right are the results of using three span values during each run of 
Supersmoother. The two top right graphs in Figure 4.4 illustrate the 
difference between robust smoothing, i.e. ALPHA= 0.0, and non-robust 
smoothing, i.e. ALPHA= 10.0. The single span value curves in Figure 
4.4 are quite similar to the smooth curves produced by LOWESS. Тһе 
reason for the similarity is that both LOWESS and Sumner are 
central smoothers, i.e. the smooth point value is the result of aver- 


aging over the points in the neighborhood. 


When three span values are used the smooth points generated by 
Supersmoother converge much faster to the underlying linear function 
than the smooth points generated by  LOWESS. Therefore, the 
Supersmoother algorithm traces very well the linearity of a data set 


with linear trends. 


When the Split Linear Fit algorithm is used with only one window 
size, the resulting curves, shown on the left-hand side of Figure 4.5, 
are not much different from the curves produced by LOWESS and 
Supersmoother. The right-hand graphs of Figure 4.5 illustrate that 
when the Split Linear Fit algorithm is given more than one window size, 
the generated smooth point values do not converge to the linear under- 
lying function as fast as Supersmoother. The smaHest window size, 
i.e. 10 and 15, in each case has a great impact on the shape of the 
smooth curve, because with few points in the window the outliers 
receive higher weights than in 'large' windows. This illustrates that 
this smoothing algorithm is designed to place more emphasis on the 


outlying data points in the data. Equation 1.4 explains that smaller 
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Figure 4.4 Test Set One: Smoothing With Supersmocther. 
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window sizes will have less bias and thus show more shape, which is 
the case shown in the right-hand plots of Figure 4.5. Therefore, the 
user's selection of the smallest window size will determine the degree of 


convergence toward the linear underlying function. 


Figure 4.6 exhibits smoother, more linear curves than were shown 
in Figure 4.5, the reason being that larger window sizes are used, i.e. 
increase of bias and decrease of the variance. The bottom graphs of 
Figure 4.6 demonstrate the effect of increasing the minimum window 
size, MNWNSZ. In Chapter II of this thesis the use of MNWNSZ is 
discussed. The author of this thesis, after using the Split Linear Fit 
for several months, recommends that the size of MNWNSZ be less than 
one-half the size of the smallest window size being used. The bottom 
left graph of Figure 4.6 illustrates the distorting effect produced when 
this recommendation is not followed. The Split Linear Fit algorithm 
does a good job of depicting data with a linear trend, but the user 
needs to understand the theory behind the Split Linear Fit, e.g. the 
window sizes had to be increased in order to produce a smoother curve, 


in order to achieve acceptable results. 


Another measure of performance that can be used in verifying the 
efficiency of a smoother is the sum of squared residuals. Table 1 
shows the sum of squared residuals for the 'best' fitting linear curves 
produced by each smoother, where 'best' is supported by a plot of the 


smooth curves shown in this chapter. 


All the fits listed in Table 1 produce a fairly straight line which is 
close to the true underlying function, Y = X. Other fitted curves do 
produce lower values of sum of squared residuals but the plotted curve 
deviates from a straight line, e.g. the plot produced by Supersmoother 
with SPAN(s) = 0.05, 0.3, 0.5 and ALPHA = 0.0 is not very straight, 
but the sum of squared residuals is 204.6112056. The decrease in the 
sum of squared residuals is due to an increase of bias. Supersmoother 
performed almost as well as the Linear Regression and LOWESS, but 
three neighborhood sizes had to be used, instead of one. The Split 
Linear Fit smoother did not do as well as Supersmoother for the 


following reasons: 
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Figure 4.5 Test Set One: 
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Figure 4.6 Test Set One: . Split Linear Fit 
Change of Smallest Window Size and MNWNSZ. 


1. Neighborhood sizes larger than those used by the other 
smoothers had to be used by Split Linear Fit before a smooth 
curve could be produced, therefore, this smoother is slower 
than the other smoothers in converging to a known underlying 
function. 

2. The sum of squared residuals value produced by the best, 
Split Linear Fit curve is the largest of all the values, there- 


fore, this curve is not as accurate as the other 'best' curves. 


va = 


TABLE 1 


TEST SET ONE: 
SUM OF SQUARED RESIDUALS OF THE BEST FITS 


Sum of Squared 
Type of Fit Residuals 


Linear Regression 205 . 94074 

LOWESS, robust, F= 0.5 205 . 92350 

LOWESS, non-robust, F= 0.5 205.82359 

А ALPHA= 10.0 205 . 84026 
SPAN(s)= 0.05, 0. | 

Split Linear Fit, MNWNSZS 50 206.5657 
WNSZ(s)- 160, 170, 180, 190 


TABLE 2 
TEST SET ONE: COMPUTER CPU CONSUMED 


CPU Consumed 
Type of Fit (in Seconds) 


Linear Regression 

LOWESS, robust, F- 0.5 

LOWESS, non-robust, F- 0.5 

Supersmoother ALPHA- 19. 0 
SPAN(s)= 5 

Split Linear Fit, OMNWNSZC. 50 
WNSZ(s)- 160, 170, 180, 190 





In Table 2 are listed the Central Processing Units, i.e. unit of time 
used by the IBM 3033 computer in processing a program, consumed 
when the smoothing techniques listed in Table 1 are used. In order to 
be consistent in the CPU measurements, each smoother was used to 
smooth the same data set and place the smoothed output in an APL 
variable. The CPU times listed in the table indicate that the advanced 
smoothers do better than most of the other smoothers, but the improve- 
ment is only in seconds. Therefore, a user trying to select between 
the smoothers should balance this saving in CPU time with the cost of 


computer and personnel time, before deriving a conclusion. 
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D. TESTING AND RESULTS----SMOOTH CURVATURE IN UNDERLYING 
FUNCTION 
The second test is designed to test how the Supersmoother and the 
Split Linear Fit algorithms perform on a data set which has an under- 
lying function with smooth curvature, i.e. the change from one point to 
the next is not abrupt. Figure 4.7 displays Test Set Two which 


consists of 200 data points generated with the following equation: 
i X е 
К COS 5 + Normal (0,1) noise, 0<X<200 . (4.4) 


The values generated by this function and used in this section are in 


tabular form in Appendix D. 


TEST SET TWO WITH N(0,1) NOISE TEST SET TWO WITHOUT N(0.1) NOISE 





Figure 4.7 Test Set Two. 


Figure 4.8 shows the curve fitting results of third degree polyno- 
mial curve fit to Test Set Two. Confidence Interval Tests on the coef- 
ficients of the equation shown on Figure 4.8 reveal that the coefficients 
are not significantly different from zero with a Confidence Level of less 


than 0.001, thus the coefficients should be accepted. 
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Figure 4.8 Test Set Two: Third Degree Polynomial Curve Fit. 


Figure 4.9 shows the results of the Spline fit, as previously 
discussed this GRAFSTAT function requires that the user enter the 
maximum sum of squared residuals value. In Figure 4.9 O indicates 
that the smooth function should have a sum of squared residuals less 


than or approximately equal to the second parameter. 


Figure 4.10 shows the smooth curves produced from using the 
Equal-Weight Moving Average smoother. As usual a larger neighbor- 
hood size, M, results in a smoother curve. As stated in Chapter I, the 
disadvantage of using the Equal-Weight Moving Average smoother is that 
smooth data points are dropped from the ends of the output data set. 
This is illustrated in Figure 4.10 where with M=60, 30 points are 


dropped from each end. 


Figure 4.11 shows the results produced by the Cosine-Weighted 
Moving Average smoother. Since this smoother is an extension of the 
Equal-Weight Moving Average, it can be seen that as the neighborhood 
size, M, is increased smooth point values are also dropped from the 
ends of the data set. In addition, this figure illustrates how the 


smooth curve converges toward the true underlying function and then 
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Figure 4.9 Test Set Two: Spline Fit. 
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Figure 4.10 Test Set Two: . 
Equal-Weight Moving Average Smoothing. 


away from this same underlying function, i.e. the variance decreases 


but bias increases beyond a certain point. 
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| Figure 4.11 , Test Set Two: 
Cosine-Weighted Moving Average Smoothing. 


The LOWESS smooth results are shown in Figure 4.12. Since Test 
Set Two appears to be non-linear, the quadratic fitting option of 
LOWESS is used. Only the robust cases are shown since the results 
are basically the same as the non-robust cases. The best fitting curve 


appears to be the smooth curve produced by using F= 0.5. 


The Supersmoother results are displayed in Figure 4.13. The 
smooth curves produced by using three span values tend to maintain 


the shape of the underlying function across most span values. 
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Figure 4.12 Test Set Two: LOWESS Smoothing. 


However, if only one span value is used, the shape of the smooth curve 
approaches a straight line as the size of the span value approaches the 
value of 1.0. The top right and the middle right graphs illustrate the 
difference between using robust smoothing, i.e. ALPHA= 0.0, and 
non-robust smoothing, i.e. ALPHA= 10.0. Therefore, it is best to use 
Supersmoother with three span values and ALPHA=0.0. 


Figure 4.14 displays a radical difference between using a single 
window size and several window sizes as input to the Split Linear Fit 


algorithm. As stated in Chapter III, the main purpose of the Split 
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Figure 4.14 Test Set Two: Smoothing With Split Linear Fit. 
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Figure 4.15 Тез Set Two: Split Linear Fit 
Change of MNWNSZ. 


Linear Fit smoother is to disclose sudden abrupt changes in a data set. 
By using a single window size, the algorithm does not produce enough 
information about the true shape of the given data set, and the result 
is the deviating smooth curves on the left side of Figure 4.14. The 
use of several window sizes produces more information about the raw 
data, and the result is smoother curves; see the graphs on the right of 
Figure 4.14. Figure 4.15 shows the effect produced when the size of 
MNWNSZ is changed and when large window are used. The smoothed 
output produced by the Split Linear Fit smoother never converges to 


the true underlying function no matter what window sizes are used. 
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The sum of squared residuals corresponding to the 'best' fitting 
curves are listed in Table 3. Each of the fits listed in Table 3 has a 
different degree of convergence toward the true underlying function, 

Y = COS(X/25). Other fits not listed in Table 3 but discussed in this 
section produced smaller sum of squared residuals, however the smooth 
curve did not resemble the true underlying function to a high degree. 
Supersmoother performs fairly well when given a set of data with a 
smooth curvature. The Split Linear Fit smoother did a poor tracing the 
true underlying curve very well, when compared to other more simple 


smoothers. 


TABLE 3 


TEST SET TWO: 
SUM OF SQUARED RESIDUALS OF THE BEST FITS 


Sum of Squared 
Type of Fit Residuals 


Third Ко ое: Curve Fit . 14846 
Spline Fit (0, 203) Eu T3 
Equal-Weight Moving Average, M- 60 . 04903 
Cosine-Weighted Moving Average, M- 61 . 38194 
LOWESS, robust, F- 0.5 ‚26 
LOWESS, non-robust, F= 0.5 . 133 
Supersmoother, ALPHA- 0. . 28674 
SPAN(s)- 0.05, 0.3, 0. 
Split Linear Fit, MNWNSZ- „51462 
WNSZ(s)= 25, 50, 100, 150 





Table 4 shows the CPU times consumed by the smoothers listed in 
Table 3. The Cosine-Weighted Moving Average smoother, in addition to 
producing a very good sum of squared residuals value, is very fast in 
generating the smoothed results. The advanced smoothers were much 
slower than the Cosine-Weighted Moving Average, but were faster than 
LOWESS. 
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TABLE 4 
TEST SET TWO: COMPUTER CPU CONSUMED 


CPU Consumed 


Type of Fit (in Seconds) 
Third Degree Eu Curve ЕШ 0.96 
Spline Fit (0 12:955 
Equal- SENE МА Average, M= 60 0.06 
Cosine- мевае Moving Average, M= 61 0222 
LOWESS, robust, F= 0.5 21.64 
LOWESS, non-robust, F= 0.5 ас 
Supersmoother ALPHA= 10.0 1.55 
SPAN(s)= 0.0 0-3, 22. 5 
Split Noe Fit MN 2з 


E. TESTING AND RESULTS----ABRUPT CHANGES IN CURVATURE IN 
UNDERLYING FUNCTION 
The third test examines the performance of the Supersmoother and 
the Split Linear Fit algorithms on a data set which includes a triangular 
function. Test Set Three is shown in Figure 4.16 ane the point values 
are displayed in table form in Appendix D. The following equation was 


used to generate Test Set Three: 


оо ‚ и 0<Х<50 
8.0 — 0.08X if 50<X<100 

Y = }_0.808 + 0.008X , if 100<X<150 (4.3) 
0.392 if 150<X<200 . 


In order to check the data set against equation 4.5 a linear regres- 
sion was done on each part of the above equation and the results are 
displayed in Figure 4.17. This figure shows that the regression equa- 
tions deviate from the true equations. A Confidence Interval test done 
on each of the coefficients indicates that the coefficients are signifi- 
cantly different from zero, therefore, the equations produced by the 


regression are accepted. 


82 а 


The curve resulting from third degree polynomial fit on Test Set 
Three is shown in Figure 4.18. A Confidence Interval test done on the 
coefficients reveals that the coefficients produced by this fitting tech- 
nique are not significantly different from zero with a Confidence Level 
less than 0.001. Thus all the coefficients in the equation shown in 


Figure 4.18 are accepted. 


TEST SET THREE WITH N(0,1) NOISE TEST SET THREE WITHOUT N(0,1) NOISE 





Figure 4.16 Test Set Three. 


A plot of the point values generated by a Spline Fit with the sum of 
squared residuals required to be no greater than 204 is shown in 
Figure 4.19. 


The smooth point values generated on two runs of the Equal-Weight 
Moving Average are plotted, and the curves are displayed in Figure 
4.20. 


The curves produced by plotting the smooth point values generated 
by the Cosine-Weighted Moving Average smoother are displayed in 
Figure 4.21. 
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Figure 4.17 Test Set Three Broken Into Four Linear Sections. 


The LOWESS smoothing results are displayed in Figure 4.22. The 
best fit is produced by the run with F- 0.3. The smooth curve almost 


coincides with the true underlying function. 


Figure 4.23 contains the plots of the smooth points produced by 
Supersmoother. As can be seen, the best fit is when three span values 
are used in the smoothing, i.e. SPAN(s)= 0.05, 0.3, 0.5, which happen 
to be starting values recommended by the Friedman and Stuetzle 
[Ref. 9: p. 9]. The smooth curve tends to depict the true underlying 
function very well. The top right graph and the middle right graph 
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Figure 4.18 Test Set Three:. 
Third Degree Polynomial Curve Fit. 
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Figure 4.19 Test Set Three: Spline Fit. 


show the great disparity between using non-robust smoothing, i.e. 
ALPHA= 10.0, and robust smoothing, i.e ALPHA= 0.0. Thus with the 


Supersmoother three span values with a ALPHA=0.0 are necessary to 


get useful results. 
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Figure 4.20 Test Set Three: . 
Equal-Weight Moving Average Smoothing. 


By examining the plots in Figure 4.24 and 4.25, it is obvious that 
the Split Linear Fit smoother produces results which are often too 
erratic and not useful. Outlying points have too much influence on the 
output produced by this smoother. The only plot without any drastic 
deviations from the curvature of the underlying function is the top left 


plot in Figure 4.25. 


Table 5 shows for the third time that in addition to not producing 
good smooth curves which depict the underlying function very well, the 
advanced smoothers do not produce sum of squared residuals values as 


good as the baseline smoothers. 


Table 6 shows that the advanced are consistently using the same low 
amount of CPU time. The LOWESS has fluctuated in CPU usage, but 
has always used the most CPU. The Cosine-Weighted Moving Average 
smoother for the second time has generated a very good sum of squared 


residuals value and has used the least amount of CPU. 
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Figure 4.21 Test Set Three: 
Cosine-Weighted Moving Average. 


F. CONCLUSIONS 


The advanced smoothers, Supersmoother and Split Linear Fit, are 
quite complex to thoroughly understand and require that the user enter 
many parameters. The interrelationship between the parameters is not 
clear, and the results are difficult to control. For example, the 
smallest neighborhood size used by the advanced smoothers has more 
influence on the output than the other neighborhood sizes, but this 


value can not be too big or the output will get distorted. In addition, 
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Figure 4.22 Test Set Three: LOWESS Smoothing. 


to the three or more neighborhood sizes that the user must think about 
and calculate, а fourth value must Бе considered, i.e. the 
Supersmoother's ALPHA value and the Split Linear Fit's MNWNSZ value. 
Changing anyone of these values in the advanced smoothers produces 
radical changes in the shape of the fitted curve, leaving the user 
confused as to which values to change and by how much. Friedman 
and Stuetzle recommend that the neighborhood sizes be between 5 and 
90 percent of N, where N is the number of points to be smoothed. They 


also claim that "savings are substantial' [Ref. 9: p. 5], i.e. in the 
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Figure 4.23 Test Set Three: 
Smoothing With Supersmoother. 
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Figure 4.24 Test Set Three: 
Smoothing With Split Linear Fit. 
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Figure 4.25 Test Set Three: Split Linear Fit, 
Change of MNWNSZ. 


time required to find a desired smoothed curve is greatly reduced. 
The SPAN values used in this thesis meet this criteria. The program 
runs fast, but the sum of squared residuals values produced are not as 
good as the values produced by the simpler, more user friendly 


smoothers, one of which uses far less CPU. 


McDonald and Owen never really give any guidance on the number 
of window sizes to use except that they used "several (typically three 


to five)" [Ref. 10: p. 2] in their testing of the Split Linear Fit 
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TABLE 5 


TEST SET THREE: 
SUM OF SQUARED RESIDUALS OF THE BEST FITS 


Sum_of Squared 
Type of Fit Residuals 


Third ES. bo Pol поща, Curve Fit 
Spline Fi 
Equal- о Moving Average, M= 60 
Cosine-Weighted Моше Average, M= 61 
LO S, robust 0. 
LOWESS, поп- bed F- 0.3 
Supersmoother ALPHA= 0 „0 

SPAN(s)= 0.05, 0.3 5 
Split een Fit MNWNSZE 

WNSZ(s)= 25, 50, 100, 150 


TABLE 6 
TEST SET THREE: COMPUTER CPU CONSUMED 


CPU Consumed 
Type of Fit (in Seconds) 


Third De гое po. ou Curve Fit 
Spline Fi 
Equal-W ак МЕЕ Average, M- 60 
Cosine-Weighted Moving Average, M= 61 
LOWESS, robust, F= 0.5 
LOWESS, non-robust E 0.5 
Supersmoother 

SPAN(s)- : 
Split MEE Fit W 

WNSZ(s)= 160 170, 180. 190 


I 


I 
> нс ООО 
1 





smoother. The smooth curves produced by this smoother never are 
consistent, i.e. follow a pattern which can be used as a guide toward 
the desired smooth curve. Figure 4.24 is a good example of this 
problem. The smooth curve is totally different from one graph to the 


next. Results may be obtained fast, but a user wants good results. 
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In conclusion, even though the advanced smoothers are fast, they 
do not perform as well as the LOWESS smoother or the faster 
Cosine-Weighted Moving Average smoother in depicting simple functional 
relationships. The consistently good sum of squared residuals values 
and smooth curves produced by these simple smoothers favor their use 


over the advanced smoothers. 
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V. EVALUATION/APPLICATION QF AHE ADVANCED SMOOTHING 


A. GENERAL 


The evaluation of Supersmoother and Split Linear Fit continues in 
this chapter. As stated in the previous chapter, a smoothing algorithm 
is used to extract the underlying relationship from a data set. 
Smoothing is especially useful if the underlying relationship is complex, 
i.e. too difficult to describe mathematically or use simple (global) least 
squares regression. The small and simple data sets in the previous 
chapter are easy to smooth since the shape of the underlying function 
is quite visible in a scatterplot of the raw data, see Figures 4.1, 4.7, 
and 4.16. Least squares regression is thus easy to apply to these data 
sets. On the other hand, the least squares method does not adequately 
smooth the data set tabulated in table form in Appendix D. A plot of 
this data is shown in Figure 1.1. The great amount of variability 
inherent in the data set causes the regression technique to be inade- 
quate, i.e. the raw data is very erratic. Most data sets collected from 
real populations/situations do not have a constant nor smooth variance, 
thus regression techniques fail to be adequate, i.e. the fitted curve 


is too smooth and the sum of squared residuals is too high. 


In this chapter the data set utilized is the daily sea-surface temper- 
atures at Granite Canyon, just south of Point Sur, California [Ref. 10]. 
The data displayed in Figure 1.1 is the first of thirteen years of sea- 
surface temperature data collected at this location. This data set defi- 
nitely does not have a constant variance, but it seems to exhibit some 
periodicity and to have some points of discontinuity, notably a very 
sudden and strong drop in temperature because of current up wellings 
in the spring. This data set was selected for final evaluation of 
Supersmoother and Split Linear Sit because of the folowing reasons: 


ihe this data set has been a subject of intense data analysis 
[Ref. 4]; 
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2. the characteristics exhibited by this data set, mentioned above, 
апа; 


3. it was easily accessible. 
B. METHODOLOGY 


The procedure followed in this chapter in the evaluation of 
Supersmoother and Split Linear Fit is described in the previous chapter 
in the Methodology section. The procedure is outlined below: 

Пе display and examine the data set to be smoothed; 

a display and examine the smooth results produced by the base- 
line smoothing techniques, i.e. the Least Squares Regression, 
the Equal-Weight Moving Average, the Cosine-Weighted Moving 
Average, and LOWESS; 

В. display and examine the smooth results produced by the 
advanced smoothers; 

4. compare these results to the results from 2 above. 

In the previous chapter the graphs showing the smooth points produced 
by any one smoothing technique were displayed together, e.g. see 
Figure 4.3 which has all the Test Set One LOWESS smoothing results. 
In this chapter it is best to display together the smoothing results that 
use equivalent neighborhood sizes, as described below: 

ДЕ. the neighborhood size in the Moving Average smoothing tech- 
nique is called M, which is equivalent to window size used in 
Split Linear Fit; 

p. the span value used by Supersmoother is equivalent to the F 
value used by LOWESS, both are computed as the ratio of the 


neighborhood size to the number of data points to be smoothed. 


This change in plot display makes it easier to subjectively decide 
the adequacy or usefulness of the advanced smoothing algorithms. The 
word 'subjectively' is used as the measure of effectiveness because, as 
mentioned before, the decision to use one smooth curve over another is 
basically based on the user's needs and desires and on the curve's 
appearance. In order to bare the comparison a more concrete statistical 
analysis, the sum of squared residuals of the different plots will be 


calculated. 
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The complete sea-surface temperature data set contains 4380 data 
points, i.e. a sea-surface temperature corresponding to each day for 
the period of March 1, 1971 to February 1983, excluding the dates of 
February 29. Only the first 671 data points which correspond to 1971 
and 1972 will be used in the evaluation in this chapter so as to have a 


manageable input data set. 


Figure 5.1 shows a scatterplot of the data set used in this chapter. 
The axis scales shown in this figure will be the standard axis scales to 
be used in all the graphs corresponding to this chapter. The vertical 
axis which is easiest to describe displays the sea-surface temperature 
range in degrees centigrade. The horizontal axis displays the day that 
the temperature was measured. The numbers shown indicate the 
Calendar date, i.e. the first digit indicates the year, assuming that the 
corresponding decade is known. Recall that the abbreviated data set 
used in this chapter was collected in the 1970's. The next three digits 
indicate the day of the year, for example '080' means the goth day of 
the year. The vertical dashed grid lines indicate the change in season 
during the year, e.g. at 1080 winter 1971 ends and spring 1971 begins. 
lhe solid tic marks indicate the end of a month. All the graphs begin 
with January. This is the reason that in Figure 5.1 the first two bins, 
January and February, are empty since the data set begins with March 
1, 1971. Finally, since the abscissa are measured in days, the neigh- 
borhood sizés used in this chapter will also be in days, e.g. a neigh- 


borhood size of 5 will correspond to a period of 5 days. 
C. TESTING AND RESULTS 


Figure 5.1 exhibits the data set that is evaluated in this chapter. 
From this graph alone, it could be deduced that there are two tempera- 
ture cycles present in the data set. The first cycle peaks near the 
end of summer 1971, and second cycle peaks just after the beginning of 
fall 1972. Another point of view that the analyst may take by looking 
at Figure 5.1 is to analyze the point distribution between the 
peaks,e.g. the data points from beginning of fall 1971 to beginning of 
fall 1972. Within this period the temperature appears to follow a cyclic 
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process, but with a smaller period, e.g. from 1355 to 2080 the temper- 
ature increased and then decreased and from 2080 to 2172 the opposite 
is reflected. Therefore, one analyst may try to depict the intra-annual 
cycle, while another may try to display the inter-annual cycle, while 
still another may try to show both cycles or something else. The anal- 
ysis in this chapter will concentrate on the intra-annual cycle as 
depicted by the advanced smoothers and compare these results to 
curves produced by previously validated and well-accepted smoothing 


techniques. 


DAILY SEA SURFACE TEMPERATURE AT GRANITE CANYON 
MARCH 1, 1971 TO DECEMBER 31, 1972 
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Figure 5.1 Data Set For Practical Application. 


Figure 5.2 shows the data divided into the two annual periods in 
order to display the intra-annual characteristics of the data. With 
these displays the intra-annual variability is more detectable and a 
different view of smoothing the data can be taken, i.e. the data can be 
smoothed to show the cyclic effect within the seasons, for example 
between 1172 and 1264. 
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DAILY SEA—SURFACE TEMPERATURE AT GRANITE CANYON, СА DAILY SEA- SURFACE TEMPERATURE AT GRANITE CANYON, CA 
MARCH 1. 1971 TO DECEMBER 31, 1971 JANUARY 1, 1972 TO DECEMBER 31, 1972 
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Figure 5.2 Data Set For Practical Application 
Divided Into Two Parts. 


Figures 5.3 through 5.8 show the smooth curves produced by the 
smoothers being compared in this chapter. A different neighborhood 
size was used to smooth the data in each case. One of the three span 
values used in the Supersmoother corresponds to the neighborhood size 
that is used by the other smoothers within a figure; the same applies to 
the Split Linear Fit. In order to properly evaluate the adequacy of the 
smoothing produced by the advanced smoothers, it is best to compare 
similar smooth curves produced by both the advanced smoothers and the 


baseline smoothers. 


As the neighborhood size increases, the produced smooth curve gets 
smoother. This effect is described by the analytical equation 1.4. In 
addition, as the neighborhood size increases, the Moving Average type 
smoothers loose more smooth data points from each end; this effect was 


described in Chapter I. 
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After each figure is a table which compares the sum of squared 
residuals corresponding to the curves displayed in the figure. This 
gives the analyst a better and more statistically supported comparison 
between the smooth output produced by the different smoothers. Іп 
addition, a table showing the CPU time used by each of the smoothers 


follows the sum of squared residuals table. 


Figure 5.3 presents the smooth curves which are produced when the 
neighborhood size is small,i.e. neighborhood size of 5. The curves are 
very erratic and somewhat unpleasant to the eye. Because some of the 
peaks are too close to each other, the short term cyclic effect is over- 
emphasized and rendered useless. In Table 7 it can be seen that the 
sum of squared residuals that correspond to the advanced smoothers are 
not as low as those corresponding to the simple smoothers. In fact the 
sum of squared residuals corresponding to the Supersmoother is almost 
three times that produced by LOWESS, which is the lowest value. 
Table 8 shows that the Cosine-Weighted Moving Average smoother is 
about three times faster than Supersmoother and about 5 times faster 


than Split Linear Fit, yet more accurate. 


In Figure 5.4 the smooth plots are not as jagged as the ones shown 
in Figure 5.3. The reason for this difference is that the neighborhood 
size used to produce the smooth curves in Figure 5.4 is twice that of 
those used to produce the smooth curves in Figure 5.3. There are 
some slight differences between each smooth curve in Figure 5.4, but 
the differences are difficult to detect. The curve produced by 
Supersmoother seems to have the least amount of jagged peaks thus 
making it easier to count the increasing and decreasing cycles within 
each season. However, Table 9 shows that the sum of squared resi- 
duals produced by Supersmoother is not as good as those produced by 
the LOWESS, either robust or non-robust smooth curve. The analyst in 
this case has the choice of deciding whether to have a good smooth 
curve with a high sum of squared residuals or a not so smooth curve 
with a low sum of squared residuals. Table 10 shows that the 


Cosine-Weighted Moving Average smoother is much faster than either of 
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TABLE 7 
SUM OF SQUARED RESIDUALS e NEIGHBORHOOD SIZE OF 


Sum of Squared 
Type of Fit Residuals 


Spline (0,100) 100. 47089 
Equal-Weight Moving Average, M= 5 84.9068 
Cosine-Weighted Moving Average, M= 5 48.0688 
LOWESS, robust, F= 0.00745 49.1153 
LOWESS, non-robust, F= 0.00745 37.3108 
Supersmoother, ALPHA= 0.0 . 1947 


16 : 
SPAN(s)- 0.00745, 0.016393, 0.0175 
Split Linear Fit, MNWNSZ- 2 © ‚ 5805 
WNSZ(s)= 


oe T3 


TABLE 8 
CPU USAGE: NEIGHBORHOOD SIZE OF 5 


CPU Consumed 
Type of Fit (in Seconds) 


Spline (0,100) 26.6 
Equal-Weight Moving Average, M= 5 0.03 
Cosine-Weighted Moving Average, M= 5 0.69 
LOWESS, robust, F= 0.00745 16.07 
LOWESS, non-robust, F- 0.00745 ‚41 


Supersmoother, ALPHA= 0.0 | О 
SPAN(s)= 0.00745, 0.016393, 0.0175 

Split Linear Fit, MNWNSZ= 2 9 
WNSZ(s)= 5, 11, 13 
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SMOOTHING WITH EQUAL=WEIGHT MOVING AVERAGE, М= 13 


SPUNE FIT, PARAMETERS (0,200) 
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TABLE 9 
SUM OF SQUARED RESIDUALS S NEIGHBORHOOD SIZE OF 


sum of Squared 
Type of Fit Residuals 


Spline (0,200) 200.51872 
Equal-Weight Moving Average, M= 13 236.86018 
Cosine-Weighted Moving Average, M= 13 13а. 92327 
LOWESS, robust, F= 0.0175 86.4623 
LOWESS, non-robust, F= 0.0175 (13993 


Supersmoother, ALPHA= 0.0 „94982 
SPAN(s)= 0.016393, 0.0175, 0.031296 


Split Linear Fit, MNWNSZ= 2 ‚ 25445 
WNSZ (sie 11, 48, 21 


TABLE 10 
CPU USAGE: NEIGHBORHOOD SIZE OF 13 


CPU Consumed 
Type of Fit (in Seconds) 


Spline (0, 200) 38.0 
Equal-Weight Moving Average, M= 13 0.04 
Cosine-Weighted Moving Average, M= 13 0.73 
LOWESS, robust, F= 0.0175 12 02 
LOWESS, non-robust, F= 0.0175 З 95 


Supersmoother, ALPHA= 0.0 ез2 
SPAN(s)= 0.016393, 0.0175, 0.031296 


Split Linear Fit, MNWNSZ= 2 emm 
WNSEs)s 11, 13, 21 
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the advanced smoothers in addition to being the most accurate 


smoother. 


With a neighborhood size of little less than a month, i.e. 21 which 
is equivalent to 21 days since time is the unit of measurement of the 
abscissa in this data set, most of the smoothers produce smooth curves 
which have lost the jagged effect at the high and low points of the 
smooth curves, see Figure 5.5. In this figure, the smooth curve 
produced by Split Linear Fit displays its tendency to follow and empha- 
size outliers. Split Linear Fit results are so robust that the peaks are 
shown pointed and not round as displayed by the other smoothers. 
This effect is caused by the edge-detection weighting scheme of Split 
Linear Fit. As shown in Table 11, the sum of squared residuals 
produced by the advanced smoothers are higher than most of the other 
smoothers, even though a very similar smooth curve is produced by all 
the smoothers. The difference between the sum of squared residuals 
value corresponding to Split Linear Fit and Supersmoother, respec- 
tively, can be explained by the tendency of Split Linear Fit to follow 
outliers more closely than Supersmoother. The Split Linear Fit 
smoother lets the raw data dictate the shape of the smooth curve, 
therefore, the difference between the raw data and the smoothed data is 
smaller for the Split Linear Fit than for the Supersmoother. Table 12 
shows again that the Cosine-Weighted Moving Average smoother is 
faster, even though the sum of squared residuals value may not be the 
best. The Supersmoother and the Split Linear Fit smoothers have 


consistently maintained their usage of CPU. 


Figure 5.6 displays the results produced by using a neighborhood 
size equivalent to almost one month, i.e. 29 days. Though still quite 
Similar, each smoother produces a visibly different smooth curve. The 
only exception is Equal-Weight Moving Average smoother which has 
suppressed the influence of the outliers. The shape of the input data 
is stil being maintained by most of the smoothers, especially Split 
Linear Fit which is designed to do so. In Table 13, the sum of 


squared residuals values corresponding to the advanced smoothers do 
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SMOOTHING WITH EQUAL=WEIGHT MOVING AVERAGE, M= 21 


SPUNE FIT, PARAMETERS (0,225) 
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TABLE 11 
SUM OF SQUARED RESIDUALS DES NEIGHBORHOOD SIZE OF 


Sum of Squared 
Type of Fit Residuals 


Spline (0,225) 225.04812 
Equal-Weight Moving Average, M= 21 331.967483 
Cosine-Weighted Moving Average, M= 21 209.753158 
LOWESS, robust, F= 0.031296 162.382 
LOWESS, non-robust, F= 0.031296 142.891 


Supersmoother, ALPHA= 0.0 21039113 
SPAN(s)= 0.0175, 0.031296, 0.04322 


Split Linear Fit, MNWNSZ= 2 236 .34410 
WNSZ(s)= 13, 21, 29 





TABLE 12 
CPU USAGE: NEIGHBORHOOD SIZE OF 21 


CPU Consumed 


Type of Fit (in Seconds) 
Spline (0,225) 43.87 
Equal-Weight Moving Average, M= 21 0.07 
Cosine-Weighted Moving Average, M= 21 0.76 
LOWESS, robust, F= 0.031296 29 56 
LOWESS, non-robust, F= 0.031296 10.03 
Supersmoother, ALPHA= 0.0 25 5. 
SPAN(s)= 0.0175, 0.031296, 0.04322 
Split Linear Fit, MNWNSZ= 2 di 
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M= 29 


SMOOTHING WITH EQUAL-—WEIGHT MOVING AVERAGE, 


SPLINE FIT, PARAMETERS (0,250) 
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TABLE 13 
SUM OF SQUARED RESIDUALS ны NEIGHBORHOOD SIZE OF 


Sum of Squared 


Type of Fit Residuals 
Spline (0,250) 249.84103 
Equal-Weight Moving Average, M= 29 411.15847 
Cosine-Weighted Moving Average, M= 29 266.83401 
LOWESS, robust, F= 0.04322 235.881 
LOWESS, non-robust, F= 0.04322 202825 
Supersmoother, ALPHA= 0.0 265.40284 

РАМ (=) = ОЗ О, 04322. 0.09091 

Split STAT Fit ME 2 211. 79171 


WNSZ(s)= 13, 29, 


TABLE 14 
CPU USAGE: NEIGHBORHOOD SIZE OF 29 


CPU Consumed 
Type of Fit (in Seconds) 


Spline (0,250) 42.55 
Equal-Weight Moving Average, M= 29 0.08 


Cosine-Weighted Moving Average, M- 29 0.81 
LOWESS, robust, F- 0.04322 34.21 
LOWESS, non-robust, F= 0.04322 1128 


EU ALPHA= 0.0 27728 
N(s)= %. 0175, 0.04322, 0.09091 


Split таат Fit р 2 ЗӨ 
WNSZ(s)= 13, 
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not deviate as much as in the three past cases from the sum of squared 
residuals values corresponding to the other smoothers. Table 14 basi- 


cally follows the same explanation of Table 12. 


If a neighborhood of two months is used, the smooth curves shown 
in Figure 5.7 are produced. Each curve is now quite different from 
the other curves. It is now quite noticeable that the Moving Average 
type smoothers have lost smooth data points at the ends. LOWESS has 
been able to maintain the shape of the input data. The Split Linear Fit 
has made a good attempt to do as well with the edge-detecting 
weighting scheme. Since Supersmoother is a central smoother and the 
neighborhood size is larger than the intra-seasonal period, the smooth 
curve produced is quite 'smooth', i.e. not jagged and abrupt. The sum 
of squared residuals values shown in Table 15 reflect the superiority of 
LOWESS over the other smoothers. The smooth curves displayed in 
Figure 5.7 substantiate even more LOWESS's superior performance. 
LOWESS has a better sum of squared residuals value and a better 
smooth curve. ‘Table 16 shows that LOWESS is almost the slowest of the 


smoothing techniques, but it has the lowest sum of squared residuals. 


The last figure, Figure 5.8, is shown basically to illustrate that the 
Moving Average smoothers have begun to deteriorate, i.e. loose too 
many smooth data points at the ends and deviate from the shape of the 
raw data. LOWESS and the advanced smoothers are still maintaining the 
general shape of the raw data, but are beginning to get too smooth. 
The Split Linear Fit smoother has made a good attempt to depict the 
outliers, even with a large neighborhood size, but the price paid is the 
return of the undesirable sharp peaks with plateau-like bases. If the 
neighborhood size is gradually increased, the resulting smooth curves 
wil change from those in Figure 5.8 to smooth sinusoidal curves, and 
eventually to straight lines. The sinusoidal curves and the straight 
lines illustrate only general features about the raw data and defeat the 
purpose of smoothing. As shown in Table 17, where the neighborhood 
size is 91 days, the advanced smoothers finally produced the low sum of 


squared residuals values. This figure was made to illustrate that the 
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SMOOTHING WITH EQUAL-WEIGHT MOVING AVERAGE, M» 61 


SPLINE FIT, PARAMETERS (0,500) 





11722 12654 1355 2080 2172 


1080 


91 vt e 0! 8 
4833 335930 М ЗУПІУНЭАМЗІ 


: 
: 
! 
1 
! 
; 
; 
! 
| 
4 
E 
; 
' 
! 
, 
! 
! 
! 
! 
2080 


JULIAN CALENDAR DATE 


SMOOTHING WITH COSINE~WEIGHTED MOVING AVERAGE, M= 61 


1355 


а 
+ 
1264 


! 
' 
' 
' 
' 
è 
' 
' 
в 
! 
‘ 
! 
' 
1 
4 
i 
t 
' 
L] 
' 
' 
' 
' 
' 
4 
ы 
! 
! 
4 
1 
-ь 
4 
) 
4 
1172 


91 vi [A 01 8 
'1N39 335939 М ЗУГИУУ3 ам 31 


JULIAN CALENDAR DATE 


ТЕ) s b 
1 
и. 
алп» шыс ge АШ #3 
~ .... ч 
= „12 
о . : 

9 “ 4 ^ = 
--------- т^ ==әр== Шеәе«і.-------------- 
се * 9 

oz ds 
+ wE e, сі 
ag г. 
ще m 
= © m m omm hom hom dom ooo = Че = «з єз === = “+ ......---«--- 8 
ог ' = 
= 5 MT 
Е CMM cnm сд у ша 5 коз а 2 
= Pas. CUTE шй z Ya €t n 
5 ts. ° Е 
o5 d с 
2 ШАА; беге t SER И +. Du Mec x 
E О м Ех "ata "t 5s 
өз ---------------------------121.Х34-- = 
КО зыта ы ee 2 
9% vi г 01 8 


'IN39 334930 NI 3401v3431 


1355 2080 2172 2264 
JULIAN CALENDAR DATE 


1264 


1172 





1080 


9t У! е! 0! 8 
'1N32 3345930 Nt 33n1v3431 


JULIAN CALENDAR DATE 


SMOOTHING WITH SPUT UNEAR FIT 


WNSZ(S)= 29, 61, 


MNWNSZ== 10 
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SMOOTHING WITH SUPERSMOOTHER, ALPHA= 0.0 
SPAN(S)# 0.04322, 0.09091, 0.13562 
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TABLE 15 
SUM OF SQUARED RESIDUALS Ad NEIGHBORHOOD SIZE OF 


sum of Squared 
Type of Fit Residuals 


Spline (0,300) 299 T9 
Equal-Weight Moving Average, M= 61 ӘЛІ 802528 
Cosine-Weighted Moving Average, M= 61 409 . 40077 
LOWESS, robust, F= 0.09091 312.995 
LOWESS, non-robust, F= 0.09091 312.263 


Supersmoother, ALPHA= 0.0 453.02390 
SPAN(s)= 0.04322, 0.09091, 0.13562 


Split Linear Fit, MNWNSZ- 2 423.35262 
WNSZ(s)= 29, 61, 91 


TABLE 16 
CPU USAGE: NEIGHBORHOOD SIZE OF 61 


CPU Consumed 
Type of Fit (in Seconds) 


Spline (0,300) 46.17 
Equal-Weight Moving Average, M= 61 0.14 
Cosine-Weighted Moving Average, M= 61 0.98 
LOWESS, robust, F= 0.09091 S176 
LOWESS, non-robust, F= 0.09091 17.44 


Supersmoother, ALPHA= 0.0 | 2728 
SPAN(s)= 0.04322, 0.09091, 0.13562 


Split Linear Fit, MNWNSZ= 2 3.8 
WNSZ(s)= 29, 61, 91 
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advanced smoothers maintain the shape of the raw data better than the 
other smoothers when larger neighborhood sizes are used in the 
smoothing. Table 18 illustrates that the advanced smoothers maintain a 
constant CPU usage, throughout the evaluation. Tables 8 through 18 
have larger CPU usage than in the previous chapter. The reason for 


this is that the data sizes are different. 
D. CONCLUSIONS 


The two advanced smoothers investigated in this thesis, the 
Supersmoother and the Split Linear Fit, generate adequate smooth 
curves. They are faster than most current smoothing techniques. 


However, their many inputs make their implementation difficult. 


Two simpler smoothers are the LOWESS and the Cosine-Weighted 
Moving Average. Both only require a single neighborhood size as 
input. This dramatically reduces the complexity of the program for the 
user. Both generate smooth curves with satisfactory results equal to 
the advanced smoothers. However, both LOWESS and the 
Cosine-Weighted Moving Average produce better sum of squared resi- 
duals values. In addition, the Cosine-Weighted Moving Average is much 


faster than either of the advanced smoothers. 


The simpler smoothers, LOWESS and the Cosine-Weighted Moving 
Average, do have some drawbacks. LOWESS is considerably slow than 
the advanced smoothers, but the disadvantage of LOWESS is only 
apparent after many runs of the programs. The speed difference for a 
single run is minor, measured only in seconds. The disadvantage of 
the Cosine-Weighted Moving Average smoother is that values are 
dropped from the ends of the output array, as illustrated in this 
thesis. The larger the neighborhood size, the more smoothed values 
are dropped, sometimes these values are important and other times they 


are not; this decision belongs to the user. 


It is the recommendation of the author that LOWESS or the 
Cosin-Weighted Moving Average smoother be used over either of the 


advanced smoothers. The advanced smoothers are considerably more 
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TABLE 17 
SUM OF SQUARED RESIDUALS о о NEIGHBORHOOD SIZE OF 


oum of Squared 
Type of Fit Residuals 


Spline (0,450) 450.54580 
Equal-Weight Moving Average, M= 91 250.83383 
Cosine-Weighted Moving Average, M= 91 463.34429 
LOWESS, robust, F= 0.13562 458.317 
LOWESS, non-robust, F= 0.13562 454.213 


Supersmoother, ALPHA= 0.0 435.71549 
SPAN(s)= 0.04322, 0.13562, 0.2489 


Split Linear Fit, MNWNSZ= 2 452.55936 
WNSZ(s)= 29, 91, 167 


TABLE 18 
CPU: USAGE NEIGHBORHOOD SIZE OF 91 


CPU Consumed 
Type of Fit (in Seconds) 


Spline (0,450) 55421 
Equal-Weight Moving Average, M= 91 0.2 
Cosine-Weighted Moving Average, M= 91 Т 
LOWESS, robust, F= 0.13562 68.04 
LOWESS, non-robust, F= 0.13562 23 O1 


Supersmoother, ALPHA= 0.0 За 
SPAN(s)= 0.04322, 0.13562, 0.2489 


Split Linear Fit, MNWNSZ= 2 3.82 
WNSZ(s)= 29, 91, 167 
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complex than these smoothers without yielding better accuracy. As 
mentioned before, the speed difference is minor, measured only in 


seconds. 
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VI. INSTRUCTIONS ON USING THE ADVANCED SMOOTHERS 
A. GENERAL 


This chapter provides detailed instructions on how to use the 
smoothing programs developed by the author of this thesis and have the 
advanced smoothing algorithms embedded in them. The Supersmoother 
algorithm is embedded in one smoothing program and the Split Linear 
Fit algorithm is embedded in another smoothing program. Any inter- 
ested person should be familiar with this chapter before attempting to 
do any data smoothing with these programs. In order to obtain good 
and fast results and understand the smoothing algorithms, it is highly 
recommended that the user read either or both of Chapters II and III, 
depending on the program to be used. Before adjusting any embedded 
parameters, it is essential that the user read the 'Technical Description' 
chapter corresponding to the program being modified. These programs 
are designed to be used on the IBM 3033 computer currently at the 
Naval Postgraduate School. The programs are written in FORTRAN 77, 
because of the need to use negative index values. The use of both of 
the smoothing programs is very similar, so both are addressed in this 
chapter. Operations peculiar to each program are addressed as sepa- 


rate paragraphs corresponding to each program. 


The smoothing programs are completely interactive, in other words, 
the user enters the data and other pre-defined parameters when asked 
by built-in queries. The user has the option of selecting the one of 
several types of output, which are: 

Па create a CMS file and place the smooth output into this newly 

created CMS file; 

да place the smooth output into an existing APL workspace within 

a newly created APL variable; 

3. create an APL workspace and place the smooth output into this 

newly created workspace, or; 


2 


4. plot the smooth output using the GRFASTAT graphics package. 


1:045 


The smoothing programs are quite flexible in letting the user decide 
where to put the final smooth output. The user can smooth any 
number of points up to 5000 data points. The data to be smoothed 
need not be in order, because the smoothing programs have an 
embedded sorting subroutine which sorts the data into chronological 
according to the abscissa values entered by the user, before sending 


the data array to the advanced smoothing algorithms. 


The programs are written in such a way that if the user makes a 
mistake, it will be announced, and the program can be restarted or 
stopped. Practically no knowledge of FORTRAN is needed to run these 
programs. If APL is to be used, it is best to understand what the 
relevant ‘workspace’ and '‘'variable' names are [Ref. 15]. If the user 
wants to use GRAFSTAT to plot the output, it is best to get familiar 
with the GRAFSTAT 'PLOT' and 'AXIS CONTROL' functions before 
attempting to use the plotting option embeded within the advanced 


smoothing programs. 
B. TERMINAL REQUIREMENTS 


If used to create a CMS data file the Supersmoothing program is 
used to can be run from any remote terminal attached to the IBM 3033 
and located within the Naval Postgraduate School. If this smoothing 
program is to be used for APL workspace creation, then an appropriate 
APL terminal must be used. If the GRAFSTAT plotting option is to be 
used, then the IBM 3277/TEK 618 graphics terminals must be used. 


Because the Split Linear Fit generates a great amount of data it 
must be run on the IBM 3277/TEK 618 graphics terminals with a memory 
capacity of at least 2 Mega-Bytes. The bigger the input data set, the 
more data storage that the Split Linear Fit smoothing program will need. 
For each point that is to be smoothed, the computer needs the capacity 
to store a matrix that has the dimensions of 9 by the number window 
sizes entered. For example, if 200 data points are to be smoothed with 
the Split Linear Fit smoothing program and 6 window sizes are entered, 


then each data point wil need a matrix of size 9 by 6. Therefore, to 


15 


run this smoothing program, the user must have available the storage 
capacity for a matrix that is 200 by 9 by 6, i.e. 10,800 bytes, plus the 
storage capacity for the raw data, the corresponding abscissa and 
computed weights, the temporary smoothed point values, and the final 


smoothed point values. 
C. INPUT DATA FILES 


In order to use either of the smoothing programs, it is required 
that the data which is to be smoothed be in a CMS file with filetype 
'data'. If the data points are not in chronological order, the corre- 
sponding abscissa, i.e. numerical order, must be in another CMS file 
with filetype 'ORD'. 


D. PROGRAM INITIALIZATION 


The advanced smoothing program packages can be obtained from 
Professor P. A. W. Lewis, Department of Operations Research, U. S. 
Naval Postgraduate School, Monterey, CA. The Supersmoothing 
program consists of the following files: 

1. SUPSMO EXEC Al; 

zn SUPSMO FORTRAN A1; 

ST SUPSMO VSAPLWS Al. 

A copy of these files is in Appendix A. The Split Linear Fit program 
consists of the following files: 

P SPTLIN EXEC Al; 

2: SPTLIN FORTRAN A1; 

3. SPTLIN VSAPLWS Al. 

A copy of each of these files is in Appendix B. It is essential that the 
three respective files be on the same disk when either smoothing 
program is to used. The EXEC file do the following operations: 

17 activates the IBM; 

2 system libraries; 

8. queries the user for the input; 

4 designates the computer storage space to be used for input and 


output; 
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loads and runs the FORTRAN file; 

executes the APL or GRAFSTAT user options; 

returns the disk to the original state, i.e. erases the TEXT 
and LOAD files, so as not to overload the disk being used. 


The Supersmoother smoothing program is invoked by typing 
'SUPSMO' and then pressing the ENTER key on the keyboard. Next 
the user must read the information displayed and comply with the 
instructions. As long as the user follows the instructions, the 
smoothing program 'SUPSMO' will produce the desired results. If any 
deviations from the requested data occur 'SUPSMO' will let the user 
know. The Split Linear Fit smoothing program is just as easy. It is 
invoked by typing 'SPTLIN' and pressing the ENTER key on the 
keyboard. Read the information on the screen, answer the questions, 
and 'SPTLIN' does the rest. 


An example of a session using SUPSMO to create the Supersmoother 
curve in Figure 5.3 is in Appendix E. A session using the Split Linear 
Fit smoothing program SPTLIN basically follows the same line of 


questions. 
РИ ӘТТРТТ ЕШ,Е5 


The smoothing programs will put the smooth output where the user 
designates unless a file already exits with that name. If a CMS file 
already exists by the name that the user wants, then the session will 
be terminated, told the reason for the termination and to restart the 
program. If the CMS file does not exist then the program continues 
normally. For APL files, the program queries the user about the status 
of the file, i.e. exiting or to be created. If the new file exists or if 


old file does not exist, then the session is terminated, the user is told 


the reason for the termination and to restart. One word of caution: 
THE SMOOTHING PROGRAMS WILL NOT WRITE OVER AN EXISTING 
APL VARIABLE!!! The program will continue running normally, but 


the data will be lost. Therefore, it is up to the user to manage the 


disk space properly and to keep note of which file contains what type 
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of smooth data. The CMS file created by the smoothing programs will 
contain five smooth data points per row. The length will depend on the 
number of points that are smoothed, i.e. the number of points smoothed 


divided by five. 


In order to put data into an existing APL workspace or create a 
new workspace, the user types in the name of the APL workspace when 
asked to do so. ‘The will verify the status of the workspace as mention 


before. If everything is satisfactory, the program continues. 


Should the user have any specific questions about the programs it 
is recommended that the 'TECHNICAL DESCRIPTION' chapter be read. 
These chapters basically follow the smoothing procedure step by step. 
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E 


APPENDIX A 
SUPERSMOOTHER PROGRAM 


SUPSMO EXEC 


The folowing computer file is SUPSMO EXEC which activates and 


runs the Supersmoother smoothing program. Chapter VI contains 
instructions on how to use this smoothing program. 
&TRACE 
Bee eb LTP 
B or TXTLIB VLNKMLIB  VALTLIB  VFORTLIB  IMSLSP  NONIMSL 
CLRSCKRN 
&TYPE YOU HAVE mor or AN ALGORITHM 
ENTE ТО SMOOTH A SET OF DATA с THE 
&TYPE ALGORITHM "SUPER SMOOTHER 
&TYPE DEVELOPED BY FRIEDMAN AND STUETZLE OF 
а STANFORD UNIVERSITY DEPT. OF STATISTICS 
&TYPE IF GRAPHICS WILL NOT BE EF DEFINE STORAGE AS 1024K 
&TYPE BY ENTERING 'DEF STOR 1024 
ENDE FOLLOWED BY 'I CMS', 
YPE THEN BY 'SUPSMO' 
&TYPE 
&TYPE DO YOU WISH TO CONTINUE? 
&TYPE ENTER Y FOR YES OR ANY OTHER KEY TO EXIT: 
EAD VAR &CONT 
CLRSCRN 
&IF &CONT МЕ У &СОТО -ЕХТТ 
&TYPE IN ORDER TO USE THIS ALGORITHM 
DE YOU MUST HAVE ON HAND THE FOLLOWING: 
&TYPE 1. FILENAME OF DATA FILE (FILETYPE DATA) WITH 
DE DATA TO BE SMOOTHED 
&TYPE 2. IF DATA POINTS ARE NOT IN CHRONOLOGICAL ORDER, 
«ТҮРЕ YOU NEED TO HAVE A FILE (FILETYPE ORDER) 
“ТҮРЕ WITH INDICES CORRESPONDING TO DATA POINTS 
PS INDICATING THE ORDER OF THE DATA POINTS. 
&TYPE 3. FILENAME OF DATA FILE WHERE SMOOTHED OUTPUT 
«ТҮРЕ WILL BE WRITTEN OR IF YOU WANT 
&TYPE TO WRITE OUTPUT INTO APL HAVE ON HAND VARIABLE 
E AND WORKSPACE NAMES THAT WILL STORE THE OUTPUT. 
«ТУРЕ 4. IF YOU WANT TO SMOOTH THE DATA USING ONLY 
«ТУРЕ ONE WINDOW SIZE, HAVE ON HAND 
pe THE DECIMAL FRACTION OF THE DATA TO BE USED. 
&TYPE 5. IF YOU WANT TO SMOOTH THE DATA USING 
&TYPE THREE WINDOW SIZES, HAVE ON HAND 
ет THE THREE DECIMAL FRACTIONS OF THE DATA TO BE 
«ТҮРЕ 
“ГҮРЕ DO YOU WISH TO CONTINUE? 
&TYPE ENTER Y FOR YES OR ANY OTHER KEY TO EXIT: 
&READ VAR &CONT 
CLRSCRN 


&IF &CONT NE Y &GOTO -EXIT 


О 


&TYPE ENTER FILENAME OF FILE WHICH CONTAINS 
&TYPE THE DATA TO BE SMOOTHED: 


&READ ARGS 

&IF &N = 0 &GOTO -TELL 
&IF &N » 1 &GOTO -TELL 
STATE &1 DATA A1 


&IF &RC NE O 
CLRSCR 
- ORDR 

&TYPE ARE DATA 


&TYPE 
&TYPE 
&READ 


&IF &CONT EQ Y 


ЛГЕН 
ӘЛЕН 
&L IPE 
TYPE 
&READ 
STATE 
&IF 


&TYPE 
&TYPE 
ТӘЛІП 
&TYPE 
S&L YPE 
«ТУРЕ 
«ТҮРЕ 


“ЕС МЕ 0 
- 0 СЕ 
«ТҮРЕ 


&GOTO -ERROR 


N 
CLRSCRN 
POINTS TO BE SMOOTHED 
IN CHRONOLOCICAL ORDER 
ENTER Y FOR YES OR № FOR NO: 
VAR &CON d 

ЖОО - 


GO 

SINCE THE DATA POINTS ARE NOT 
IN CHRONOLOGICAL ORDER, THEREFORE 
ENTER FILENAME OF FILE (FILETYPE ORDER) 
THAT CONTAINS ORDER INDICES 
VAR &ORD 
&ORD ORDER A1 
&GOTO - ERROR 
LEN 

HE DATA YOU WANT TO SMOOTH IS IN &1 DAT 
WHERE eo WANT TO WRITE THE SMOOTHED OUTPUT? 
«ТҮРЕ YOU CAN ERO du] SMOOTHED OUTPUT 
IF YOU ARE LOGGED ON A TE oe AL 
THAT CAN ACCESS GRAFSTAT E. HAVE 2M OF STORAGE 
ВОГ ТЕ ОТР. БЕ РОВЕР" IN AN APL VARIABLE 
ENTER ы OR CMS: 


&READ VAR 

«ТЕ «РГА EQ. APL &GOTO -AP1 
КУРЫ 

CLRSCRN 


THE SMOOTHED OUTPUT WILL BE WRITTEN 
TO A CMS FILE (FILETYPE DATA) 

ENTER ONLY THE FILENAME YOU WANT 

TO res THAT CMS FILE: 


VAR &F 
THE SMOOTHED QUTPUT WILL BE WRITTEN 
TEM THE CMS FILE &FN DATA 


&G (С) 
“APTAS YLE 


S&L IFE 
&TYPE 
&TYPE 
GUY TE: 
&READ 
&TYPE 
РЕ 
&READ 


3/83 SUM E 
«ТҮРЕ 

THAT WILL CONTAIN &AÀ : 
D VAR &WKS 


NOT USING THE NAME OF THE FILE 
WITH THE INPUT DE &1 

ENTER THE NAME OF THE APL VARIABLE 
E WILL STORE. THE QUT EDS 


&A 
DO. YOU WANT TO PLOT THE OUTPUT? 
ENTER Y FOR YES OR N FOR NO: 


VAR а 
&COTO -PLOT 
THE NAME OF THE APL WORKSPACE 


IS &WKS AN EXISTING WORKSPACE OR A NEW WORKSPACE? 
ENTER O FOR EXISTING OR N FOR NEW: 
d &AGE 


- COM 

&TYPE CAN YOU ACCESS 2M OF STORAGE 
ON THIS DISK d 1)? 
UE Y FOR YES OR N FOR NO: 


< 
Мы?! 


R &GRF 
«ТЕ СВЕ EQ N &COTO =- STRE 
&FN TE 


-COM CLRSCRN 
&TYPE PLEASE READ THE FOLLOWING INSTRUCTIONS VERY CAREFULLY 
«TYPE ARE YOU READY TO START THE SUPER SMOOTHING PROGRAM? 


«СҮРЕ 


ENTER Y FOR YES OR ANY CTHERVKEY SO 2x7, 
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&READ VAR &O 

CLRSCRN 

&IF &O NE Y &GCOTO -EXIT 

&QLYPE о WAIT THE SMOOTHING PROGRAM IS BEING COMPILED 


FI 05 CLEAR 

FI 06 CLEAR 

FI 07 CLEAR 

FI 08 CLEAR 

EOS CLEAR 

FORTVS SUPSMO Буг (72:50 

FIL Ou DISK &O ORDER 

FIL 07 DISK &1 RDA TA 

FIL 08 DISK &FN DATA (RECFM FBA LRECL 80 BLKSIZE 800) 
&TYPE PLEASE WAIT SMOOTHING PROGRAM IS BEING LOADED 
LOAD SUPSMO (START 


CLRSCRN 
ERASE SUPSMO LISTING 


P 
&IF &PLA EQ CMS &GOTO -EX 
N &GOTO -NGRF 
CP TERMINAL APL ON 
&STACK )LOAD SUPSMO 
&STACK a D 


&STACK &F 

&STACK DATA 

&STACK N 

&5ТАСК &Д + ,&A 
&STACK &1 <«CMSREAD 
&STACK &1 

&STACK DATA 

&STACK N 

DOCK &1 € ,&1 
&STACK SAVE 


) 
&TYPE ***x*xPLEASE WAIT, LINKING TO GRAFSTATRRKKKKKKKKKKK KK KK 
&STACK )LOAD GRAFSTAT 
&STACK DUM «CMS 'CLRSCHRN! 
&STACK i SUPSMO 
&STACK S 


Q 
Сл 
М 
PS 


&STACK &À <CMSREAD 
«ЕМ 
&STACK л 


CLEAR 
&IF &ACE EQ O &STACK ШО &W KS 
N &STACK )WSID &WKS 
&STACK SAVE SUPSMO &A 


OFF HOLD 
PL 


-DRP ERASE &FN DATA x 

CP TERMINAL APL ON 

&STACK )LOAD SUPSMO 

&STACK JERASE &A 

&STACK JERASE &1 

&STACK )SAVE 

&STACK )OFF HOLD 

EXEC APL 

- EX UM YOU HAVE FINISHED 

&EXIT 100 

ОЛЕР TYPE YOU HAVE ENTERED TOO MANY OR 
&TYPE NOT ENOUGH ENTRIES ABOUT DATA FILE 
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ЕЕ YOU NEED TO BEGIN AGAIN BY ENTERING 
&TYPE SUPSMO 
&EXIT 100 


&COTO - EX 

- ERROR &TYPE ABOVE ENTERED Ad DATA 
&TYPE DOES NOT EXIST ON YOUR A-DISK 
&TYPE CHECK YOUR FLIST AND 

&IYPE THEN BEGIN AGAIN BY ENTERING 


«ТҮРЕ SUPSMO 


101 
- EXIT &TYPE YOU HAVE FORCED AN EXIT ON THIS SMOOTHING EXEC 
&TYPE IF YOU WISH TO BECIN ACAIN ENTER 


&TYPE SUPSMO 
&EXIT 102 


2. SUPSMO FORTRAN 


The folowing file is SUPSMO FORTRAN which does the actual 
smoothing of a data set. The subroutines SUPSMU and SMOOTH of the 
folowing FORTRAN program were developed by Friedman and Stutzle 
[Ref. 9] as stated in Chapter I. 


С 
C READ SUPSMO EXEC FILE BEFORE USING THIS FILE. 


| Фо до ооо ооо ооо ово с волос ао со Зое сора во осо роса ооо по 


THIS PROGRAM READS THE INPUT DATA, Y(N) VARIABLES FROM THE FILE 
WATER DATA A1 AND THEN USES THE INTERNAL SUPER SMOOTHING SUBROUT.* 

IN ORDER TO SMOOTH THE INPUT DATA. 

THE SPANS CAN BE CHANGED BY ENTERING DESIRED SPANS ON THE VERY У 
LAST LINE OF THIS FILE. Е 


X e ft, fi; AA RARER ARAMA ARBAB ARAMA ie ie e he s ie he ie e she he Die e ie sie 9.9.9.9... 


INPUT: 
N : NUMBER OF OBSERVATIONS (X,Y - PAIRS) 
X(N): ORDERED ABSCISSA VALUES 
Y(N): CORRESPONDING ORDINATE (RESPONSE) VALUES 
WIN) : WEIGHT FOR EACH (X,Y) OBSERVATION 
IPER : PERIODIC VARIABLE FLAG 
IPER=1: XIS ORDERED INTERVAL VARIABLE 
IPER=2: X IS A PERIODIC VARIABLE WITH VALUES 
IN THE RANGE (0.0, 1.0) AND PERIOD 1.0 
SPAN: SMOOTHER SPAN (FRACTION OF OBSERVATIONS IN WINDOW). 
SPAN —0.0: AUTOMATIC (VARIABLE) SPAN SELECTION 
ALPHA: CONTROLS HIGH FREQUENCY (SMALL SPAN) PENALTY 
USED WITH AUTOMATIC SPAN SELECTION (BASE TONE CONTROL) 
(ALPHA.LE.0.0 OR ALPHA.GT.10.0: NO EFFECT) 
OUTPUT: 
18 „SMON: SMOOTHED ORDINATE (RESPONSE) VALUES 
NL (7): INTERNAL WORKING STORAGE 
E: 


FOR SMALL SAMPLES (N < 40) OR IF THERE ARE SUBSTANTIAL SERIAL 
CORRELATIONS BETWEEN OBSERVATIONS CLOSE IN X - VALUE, THEN 

A PRESPECIFIED FIXED SPAN SMOOTHER (SPAN > 0) SHOULD BE 
USED. REASONABLE SPAN VALUES ARE 0.3 TO 0.5. 


ООООООООООООООООООООООООООООООООО 
= 


REAL*4 Y(5000),X(5000),S MO(5000),WW(5000),S PAN, ALPHA, SC(5000,7) 
REAL*4 ACVR(5000), TPANS(3) 
INTEGER IR(5000),K,N,IPER,WEI,ODR 
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18 
19 


12 


13 
15 
14 


20 


17 
7 
10 


DOUBLE PRECISION WT,FBO,FBW,XM,YM,TMP,VAR,CVAR,A,H,SY 
COMMON /CONSTS/ BIG,SML,EPS 
WRITE(5, 1) 
FORMAT(1X,'ENTER THE NUMBER OF DATA POINTS TO BE SMOOTHED— 
*INTEGER VALUE') 
READ(6,*)N 
DO 19 1=1,N 
Wil) = 1. 
WRITE(5, 12) 
FORMAT(1X,’ARE THE INPUT DATA POINTS IN CHRONOLOGICAL 
*ORDER?' /,1X,' ENTER 0 FOR NO OR 1 FOR YES’) 
READ(6,*)ODR 
IF(ODR.EQ.1)GO TO 13 
READ(4,*)(X(1),! = 1,N) 
GOTO 14 
DO 151=1,N 
Х() = FLOAT() 
CALL FRTCMS(’CLRSCRN ’) 
WRITE(5,5) 
FORMAT(1X,’ENTER 1.0 IF YOU DESIRE TO USE ONLY ONE SPAN VALUE’, 
*/,/ENTER 0.0 IF YOU WANT TO USE THREE SPAN VALUES’) 
READ(6,*)SPAN 
CALL FRTCMS(’CLRSCRN ’) 
IF(SPAN.EQ.1.0) THEN 
WRITE(5,8)N 
FORMAT(1X,’ENTER THE SPAN VALUE TO BE USED’,/,1X,’FRACTION OF’,I5, 
*'. E. A REAL NUMBER BETWEEN 0.0 AND 1.0’) 
READ(6,*)SPAN 
ALPHA = 0.0 
ELSE 
WRITE(5,2)N 
FORMAT(1X,'ENTER THE LOWEST SPAN VALUE: ,/,1X, FRACTION OF’, 
“15,” .E. A REAL NUMBER BETWEEN 0.0 AND 1.0’) 
READ(6,*)TPANS(1) 
WRITE(5,3)N 
FORMAT(1X,'ENTER THE MIDDLE SPAN VALUE" /,1X,' FRACTION OF’ 
* 15,71.Е. A REAL NUMBER BETWEEN 0.0 AND 1.0’) 
READ(6,*)TPANS(2) 
WRITE(5,4)N 
FORMAT(1X,’ENTER THE HIGHEST SPAN VALUE:’,/,1X,’F RACTION OF’ 
* 15, I.E. A REAL NUMBER BETWEEN 0.0 AND 1.0’) 
READ(6,*)TPANS(3) 
CALL FRTCMS(’CLRSCRN ’) 
WRITE(5, 16) 
FORMAT(1X,’IF ONE OF THE SPAN VALUES IS SMALL' /, 
“| E. RESULTS IN A SMALL WINDOW SIZE (10 OR LESS)’,/, 
"YOU MAY WISH TO ADJUST THE SMOOTH CURVE ROBUSTNESS’ /, 
"BY ENTERING A REAL NUMBER GT 0.0 BUT LT 10.0’,/, 
"FOR NO ROBUST ADJUSTMENT ENTER 0.0’,/, 
"OR COMPLETE ROBUST ADJUSTMENT ENTER 10.0" /, 
"ENTER YOUR CHOICE") 
READ(6,*)ALPHA 
CALL FRTCMS(’CLRSCRN ’) 
ENDIF 
WRITE(6,20) 
FORMAT(1X,’****PLEASE WAIT SMOOTHING PROGRAM NOW RUNNING*****’) 
READ(7,*)(Y(1),1 =1,N) 
IF(ODR.EQ.1)GO TO 17 
CALL SORTER(X,W,Y,N) 
IPER=1 
IF(X(N).EQ.1.0)IPER = 2 
CALL SUPSMU(N,X,Y,W,IPER,SPAN,ALPHA,SMO,SC,TPANS) 
WRITE(8,10)(SMO(I), = 1,N) 
FORMAT(2X,5(F12.6,2X)) 
STOP 
END 


NI USUS RRAAARAAMARKARAAAARARAARARNARAANARARANASNEESASAMAN 


ААА ААА ААА АЛЛА КАЛАЛАЯ 


SUBROUTINE SUPSMU(N,X,Y,W,IPER,SPAN,ALPHA,SMO,SC,TPANS) 
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_ DIMENSION X(N), Y(N),W(N),SMO(N),SC(N,7), TPANS(3) 
COMMON /CONSTS/ BIG,SML,EPS 
IF (X(N).GT.X(1))GO TO 30 
SY =0.0 
SW=SY 
DO 10 J=1,N 
SY =SY +W(J)*Y(J) 
SW -SW--W(J) 
10 CONTINUE 
A — SYISW 
DO 20 J=1,N 
SMO(J) =A 
20 CONTINUE 
RETURN 
30 1+М/4 
Ј=34 
SCALE=X(J)-X(I) 
40 IF(SCALE.GT.0.0)GO TO 50 
IF(J.LT.NJJ - J - 1 
IF(I.GT.1) — 1-1 
SCALE =X(J)-X(I) 
GO TO 40 
50 VSMLSQ=(EPS*SCALE)**2 
JPER =IPER 
IF(IPER.EQ.2.AND.(X(1).LT.0.0.OR.X(N).GT.1.0))JPER = 1 
IF(JPER.LT.1.OR.JPER.GT.2)JPER - 1 
IF(SPAN.LE.0.0)GO TO 60 
CALL SMOOTH(N,X, Y, W,SPAN,JPER, VSMLSQ,SMO,SC) 
RETURN 
60 р0701=1,3 
CALL SMOOTH(N,X, Y, W,TPANS(I),JPER, VSMLSQ,SC(1,2"1-1),SC(1,7) 
CALL SMOOTH(N,X, SC(1,7), W, TPANS(2),-JPER, VSMLSQ,SC(1,2"1),H) 
70 | CONTINUE 


) 


DO 90 J=1,N 
RESMIN — BIG 
DO 80121,3 


IF(SC(J,2*1).GE.RESMIN)GO TO 80 
RESMIN — S(J,2*1) 
SC(J,7) = TPANS(I) 
80 CONTINUE 
IF(ALPHA.GT.0.0.AND.ALPHA.LE. 10.0.AND. RESMIN.LT.SC(J,6))SC(J,7) 
= §C(J,7) + (TPANS(3)-SC(J,7))*AMAX1(SML,RESMIN/SC(J,6))**(10.0- 
* — ALPHA) 
90 CONTINUE 
CALL SMOOTH(N,X,SC(1,7),W, TPANS(2),-JPER, VSMLSQ,SC(1,2),H) 
DO 110 J=1,N 
IF(SC(J,2).LE. TPANS(1))SC(J,2) = TPANS(1) 
IF(SC(J,2).GE. TPANS(3))SC(J,2) = TPANS(3) 
F = SC(J,2)-TPANS(2) 
IF(F.GE.0.0)GO TO 100 
F —-F/(TPANS(2)-TPANS(1)) 
SC(J,4) =(1.0-F)*SC(J,3) + F*SC(J,1) 
GO TO 110 
100 F = F/(TPANS(3)-TPANS(2)) 
SC(J,4) =(1.0-F)*SC(J,3) + F*SC(J,5) 
110 — CONTINUE 
CALL SMOOTH(N,X,SC(1,4), W, TPANS(1),-JPER,VSMLSQ,SMO,H) 
RETURN 
END 


тяни кии SERA OAR eee ee eee eee 


С 
(;^^^^АлААлЛАААААКААКАААААДАХААКААКАКХКАААХАХАХААКААААААААААКАКАААА КАА АСА 
SUBROUTINE SMOOTH(N,X, Y,W,SPAN, IPER, VSMLSQ,SMO,ACVR) 
DIMENSION X(N),Y(N),W(N),SMO(N),ACVR(N) 
INTEGER IN,OUT 
DOUBLE PRECISION WT,FBO,FBW,XM,YM, TMP,VAR,CVAR,A,H,SY 
XM =0.0 
YM=XM 
VAR=YM 
CVAR=VAR 
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10 


20 


30 


40 
50 


60 


70 
80 


90 


FBW=CVAR 
JPER = IABS(IPER) 
IBW —0.5*SPAN*N + 0.5 
IF(IBW.LT.2))BW = 2 
IT=2*IBW+ 1 
DO 20 1 1,IT 
J=| 
IF(JPER.EQ.2)J = l-IBW-1 
XTI 2 X(J) 
IF(J.GE.1)GO TO 10 
J=N+J 
XTI = X(J)-1.0 
МТ = W(J) 
FBO=FBW 
FBW=FBW+WT 
ХМ = (ЕВО*ХМ 4- WT*XTI/FBW 
YM =(FBO*YM + WI*Y(J))/FBW 
TMP =0.0 
IF(FBO.GT.0.0) TMP = FBW*WT*(XTI-XM)/FBO 
VAR =VAR + TMP*(XTI-XM) 
CVAR =CVAR + TMP*(Y(J)-YM) 
CONTINUE 
DO 70 J=1,N 
OUT =J-IBW-1 
IN=J+IBW 
IF((JPER.NE.2).AND.(OUT.LT.1.OR.IN.GT.N))GO TO 60 
IF(OUT.GE.1)GO TO 30 
OUT=N+ OUT 
XTO = X(OUT)-1.0 
XTI = X(IN) 
GO TO 50 
IF(IN.LE.N)GO TO 40 
IN =IN-N 
XTI = X(IN) + 1.0 
ХТО = X(OUT) 
GO TO 50 
XTO = X(OUT) 
XTI = X(IN) 
WT = W(OUT) 
FBO=FBW 
FBW=FBW-WT 
TMP =0.0 
|F(F BW.GT.0.0) TMP = FBO*WT*(XTO-XM)/FBW 
VAR = VAR-TMP*(XTO-XM) 
CVAR =CVAR-TMP*(Y(OUT)-YM) 
XM =(FBO*XM-WT*XTO)/FBW 
YM =(FBO*YM-WT*Y(OUT))/FBW 
WT -- W(IN) 
FBO=FBW 
FBW=FBW+WT 
XM =(FBO*XM + WT*XTI)/FBW 
YM = (FBO*YM + WT*Y(IN)/FBW 
TMP =0.0 
IF(FBO.GT.0.0)TMP = FBW*WT*(XTI-XM)/FBO 
VAR =VAR + TMP*(XTI-XM) 
CVAR =CVAR + TMP*(Y(IN)-YM) 
A=0.0 
IF(VAR.GT.VSMLSQ)A =CVAR/VAR 
SMO(J) =A*(X(J)-XM) + YM 
IF(IPER.LE.0)GO TO 70 
Н = 1.0/FBW 
IF(VAR.GT.VSMLSQ)H = H + (X(J)-XM)**2/VAR 
ACVR(J) = ABS(Y(J)-SMO(J))/(1.0-W(J)*H) 
CONTINUE 
J=1 
JO=J 
SY =SMO(J)*W(J) 
FBW=W(J) 
IF(J.GE.N)GO TO 100 
IF(X(J + 1).GT.X(J))GO TO 100 
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J=J+1 
SY =SY +W(J)*SMO(J) 
FBW =FBW+W(J) 
IF(J.LT.N)GO TO 90 

100 IF(J.LE.JO)GO TO 120 
SY =SY/FBW 
DO 110 |= ЈО,Ј 

SMO(l)=SY 

110 — CONTINUE 

120 J=J+1 
IF(J.LE.N)GO TO 80 
RETURN 
END 


(QAAABRARASENKAAAAERANNSARRRRAXARA ARA ARRA NAR E 


Фи ооо ореола ЗЕ строе ое ald dll ght 


SUBROUTINE SORTER(X,W.Y,N) 
REAL*4 X(N),W(N), Y(N),D(5000) 
INTEGER N,KEY(5000) 

0051=1,№ 
5 KEY(I) =I 
CALL SHSORT(X,KEY,N) 
DO 11=1,N 
1 0(1) =У(1) 
DO 21=1,N 


RETURN 
END 


Фо оо ава сбор нос лора со оба Ија 5 А ИАТА 


С 

eee eee a ene Re ee ee ee at ge a a ee cee ee ee 
BLOCK DATA 

С 


С т 2 77 - 


THIS SETS THE COMPILE TIME (DEFAULT) VALUES FOR VARIOUS 
INTERNAL PARAMETERS: 
BIG : A LARGE REPRESENTATIVE FLOATING POINT NUMBER 
SMALL : ASMALL NUMBER. SHOULD BE SET SO THAT (SML)**(10.0) 
DOES NOT CAUSE FLOATING POINT UNDERFLOW 
EPS : USED TO NUMERICALLY STABILIZE SLOPE CALCULATIONS FOR 
RUNNING LINEAR FITS 
THESE PARAMETER VALUES CAN BE CHANGED BY DECLARING THE RELEVANT 
LABELED COMMON IN THE MAIN PROGRAM AND RESETTING THEM WITH 
EXECUTABLE STATEMENTS. 


(0000003 шо ж ete ey ee ee oe ee ee ee ee es ee o oe mm 





See a ET рее 


ОООООООООООООО 


COMMON /CONSTS/ BIG,SML,EPS 
DATA BIG,SMLEPS /1.0Е20,1.0Е-7,1.0Е-3/ 
END 


3. | SUPSMO VSAPLWS 


The following two APL functions are used in conjuction with the two 


files listed above. They were developed by the author of this thesis, 
and are the main APL functions within the APL workspace SUPSMO. 
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The first APL function links the user of the smoothing program with 
GRAFSTAT and gives a user familiar with GRAFSTAT the opportunity 
to proceed into GRAFSTAT where a greater variety of graphic functions 
are available. 


V ST RT;C 
DUMeCMS 'CLRSCRN' 
THE ENTIRE DATA FILE THAT YOU WANTED SMOOTHED HAS 
TRANSFERRED! 
'TO THIS WORKSPACE SO THAT YOU MAY BE ABLE TO PLOT 


'THE SMOOTHED AND UNSMOOTHED DATA. 
THE UNSMOOTHED DATA IS IN THE VARIABLE WITH THE SAME 


Sis 
THE DATA FILE THAT YOU HAVE YOUR INPUT DATA IN.' 
бо: 


p uU WISH TO.GO ТЇ\ТО АРЬ OR CONTINUE?! 
MENTER ПО ЛО АРНО ОНО ОСМЕ ТИДЈЕ! 


= 


ty 
нат 


>13%Сх8 

t YOU WILL BE SENT TO APL AFTER YOU HAVE READ THIS 
ЛДІ TEXT. 
"uS SAFTER YOU HAVE FINISHED WORKING IN APL AND WISH TO 


ENTER PLOTERx**x*xxNOTICE THAT PLOTER HAS ONLY ONE T' 


МОН ЕМТЕК 0 АСАТМ! 
Се 


сеге и лг и ој > ој = ој== 
МӘН КАҒА о оо У Осло бм 
«jr Ou 00-10) 01€) E јој Мо О У (ји) 


LJL JL JL JL JL UA 3L C LL LL 


>C 
БОЛЕН 


The next APL function creates the APL variables to be used in the 
GRAFSTAT 'PLOT' screen. This plotting option is made available to the 
user through the above APL function. The user can use this APL 
function to do the plotting or use the GRAFSTAT graphics functions. 
A user need not fully understand how to use the GRAFSTAT plot 
screen in order to use this function. Several examples are shown with 
each requested entry so that the user can see what the entry should 
look like. 
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V 

PLOTER; :00М:Р;5У:71:72;ТР;Х2;Х0;Х5:Х2;ХҮ;ХУ;У2:10:/Ү53Ү7; 
[1] рим: 58 TCLRSCRN! 

[2] TYOU HAVE ACTIVATED THE PLOTTING FUNCTION' 

[3] ҮТ IS ASSUMED THAT THE USER IS FAMILIAR WITH THE 
GRAFSTAT PLOT FUN 

СЧ] 'AND THE AXÍS CONTROL FUNCTION! 

[5] 'IF YOU RECEIVE (MAKE) AN ERROR MESSAGE DO THE 
FOLLOWING' 

[6] '1. ENSURE THAT VM READ IS DISPLAYED IN LOWER RIGHT 
CORNER (OF SCREEN 

[7] 25 PRESS THE ENTER KEY' 

[8] '3. ENT PAGE! 

[9] '70 UNDERSCORE A LETTER HOLD THE APL/ALT KEY DOWN AND 
PRESS THE LETTER' 

[10] 'FHE PLOTTING FUNCTION WILL RESTART AT THE BEGINNING' 


I 


V 


зї 


I 


"TRE PLOTTING FUNCTION CAN BE EXITED AI ANY МЕРЕ 
ШЕ BY ENTERING! 


AT ANYTIME THAT YOU EXIT THE UAE FUNCTION! 
'YOU WILL BE IN THE CRAFSTAT WORKSPACE! 

'IF YOU WISH TO RETURN TO CMS ENTER! 

1 | JOFF HOLD' 


1 
1 1 


LB:'ENTER X ER (ENCLOSED IN QUOTES), IF 
RING MORE THAN ONE VARIA 

oe REEL UE ИЕ WITH " SEMTCOLON AND USE QUORBES! 
XV<QN 
DUM-CMS 'CLRSCRN' 
TEWIER Y VARIABLE(S) (ENCLOSED IN QUOTES AND MUST ВЕ 
AME LENGTH A t 
P ENTERING MORE THAN ONE VARIABLE, SEPARATE WITH 


'AND REMEMBER TO USE QUOTES ENCLOSING ENTIRE STRING! 
1E G. ОЕА Уо ыш 


(c LLL Ry E L 


Су 
О 
Еч 


N 
FENTER оя INDICATING TYPE(S) OF PLOT; 0- 
ZEE 0 OR T Ok 0O. 1 Ок 9 ОТК ТТЫ 


<N 
С рег па 
NTER TYPE OF SYMBOL CORRESPONDING TO EACH SYMBOLS 


PLOT (IN QUOTES)! 
PEG. Ut, T1 ORENT с. 


HB 
|| 
Еч 
ЊУ 
= 
[т] 
Q 
= 


SY<Q 
>(0%/ТР)з 0)/LP 
L1: ' ENTERA ш INDICATING TYPE(S) ОЕ БТМЕ5: 1-50 


c -J00 ftbtoNPOtU00 Oo роньковооздфаосгою- 4 


LLJL IL JL Dy челек ы) he LLL hyd 
* 


~ 3=DASH 

КЕ са Е а 3 OR 1 3 OR ANY OTHER COMBINATION OR LINE 
S IN CBAPSTAT! 

гре < 
LP:TL« 


= (CBEP )24)/L2 
L2: DUM<CMS 'CLRSCRN! 


Е SCALE OF X-AXIS (IN QUOTES) OR P (IN QUOTES) 
PREVIOUS SCALE | 
1 (E.G. ТЕГІМ" OR ЕТ МТМ ХМАО ! 
J XS«N 


Сук к EO E. E.E.EIEOFIGGOFQOCOICOILCODSTO]ICOLCISNNNNÜINAININININNEÁEI|WIB6Bm$mÍBIB5$8dÍuCma 


сомо (л FE COND. OO (ОЮ 00 з Оз I O1 E COND IA O cO O0 xj -4 6) лу E COND ES ОФК 


VENTER SCALE OF Y-AXIS (IN QUOTES) OR P (IN QUOTES) 
по SCALE! 


ОТМОК GIN iN eye ee 


<Ñ 
TENTER. THE PLOT HEADER (IN QUOTES) OR EMPTY QUOTES' 
.G. '"ТТТЬЕ!! ОВ | 

77:8 

DUM<CMS 'CLRSCEN' 

TENTER X-AXIS LABEL (IN QUOTES) ОВ! 

„А PAIR OF EMPTY QUOTES FOR NO LABEL OR TO USE AXIS 


ТЕО. ТУ БАВЕ КА и 
XL«N 
TENTER Y-AXIS LABEL (IN QUOTES) OR! 
Е s PAIR OF EMPTY QUOTES FOR NO LABEL OR TO USE AXIS 
Uc E.G. LT RACES To Ones ae | 
N 


у= 
TDO YOU WANT TO RUN THIS PAGE?! 
E O FOR NO OR 1 FOR YES! 
«= 
DUM<CMS 'CLRSCRN' 
>L3+4x{C0=0) 
РЕ et 
< 0100 


өз 
О 
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6 'PLEASE WAIT RUNNING РАСЕ' 

7 RUN PAGESAM 

7 TDO YOU WANT TO EXIT THIS FUNCTION?! 

7 ШЕТЕН O FOR NO ORI FOR YES)! 

7 CO<N 

7 *(COS1)/LE 

7 "рО YOU WANT TO RESTART THIS FUNCTION?! 

И Boone 0 FOR NOTOR 1 FOR PES! 

Z DUM<CMS 'CLRSCRN' 

7 РАС СЕЛ 

8 'THE ONLY THING LEFT TO DO IS THE AXIS CONTROL! 

D E uu THE PARTIAL PLOT THAT YOU HAVE JUST FINISHED 

8 "ENTER A 3 ELEMENT VECTOR FOR PARTIAL PLOTI' 

a En ELEMENT, 1(0): LINES AND SYMBOLS ARE (NOT) SHOWN 
к^” ELEMENT, 1(0): HEADER AND AXES ARE (NOT) SHOWN ON 
К Onn. 1(0): AXES, GRIDS, AND GRID LINES ARE 

Б.с. 1 1 O WILL SHOW EVERYTHING ON GRAPH EXCEPT AXES 
GRID LINES! 


TENTER A 4% ELEMENT VECTOR FOR AXES AND CRID ON AES 
Е Вл O0 = BOTTOM, 2 = TOP, OR 20 AT 


B ЕМЕ eae I ЛЕ), 3 = RIGHT, OR 21 + ДТ 


"3RD ELEMENT, VERTICAL GRID LINES: ВЕЛО о 
ТТЕР, 2 = ооо 
ТН ELEMENT, HORIZON. GRID LINES: 0O=NŅNO0 GRID, 
ТТЕР, ОЕ ез о 
2 2 WILL DISPLAY AXIS AT TOP AND LEFT AND 
D р, МЕ! 


L8:'PLEASE WAIT RUNNING PAGE' 
RUN PAGESAM 
LA:DUM<CMS 'CLRSCRN'! 
'ENTER X-AXIS TIC MARKS LOCATION VECTOR’ 
'OR ENTER 0 FOR STANDARD TIC MARKS' 
"ОВ ENTER 1 FOR NO TIC MARKS! 
'E.G. 1 5 11 OR A VECTOR NAME OR O OR 1' 
11,8 11 WILL SHOW TIC MARKS AT X=1, Х=5, АМР Х=11' 
<- 
TENTER X-AXIS SYMBOLS (IN QUOTES)! 
'OR ENTER 0 WITHOUT 200755 FOR STANDARD SYMBOLS' 


11 


'OR ENTER 1 WITHOUT QUOTES FOR NO SYMBOLS' 
ee Е Е ВОТ А ТЕСТОВ NAME OK O OR 1' 
<= 
ТЕМТЕК X- AXIS SYMBOLS LOCATIONS VECTOR' 
'OR ENTER O FOR SYMBOLS AT DEFAULT LOCATIONS OR NO 


LS! 
ЕАС 18 ОК A VECTOR NAME ОК 0! 
Б) аа WILL SHOW 1970 AT Xs6 AND 1971 AT Xz18' 


Х0< 
DUM<CMS 'CLRSCRN' 
TENTER Y-AXIS TIC MARKS LOCATION VECTOR' 
'OR ENTER О РОВ STANDARD TIC MARKS: 
'OR ENTER 1 FOR NO TIC MARKS' 
'E.C. 1 0 1 OR A VECTOR NAME OR Q OR 
oie 1 WILL SHOW TIC MARKS AT Ys 1, 
<- 
TENTER Y-AXIS SYMBOLS (IN QUOTES)' 
'OR ENTER 0 WITHOUT Scores FOR STANDARD SYMBOLS' 


1 


1 
Y=0, AND Y=1'! 


'OR ENTER 1 WITHOUT QUOTES FOR NO SYIMBOLS! 
ШОО ОИ ПТ'' ОК VECTOR NAME OR O OR 1' 


YIYI«-N 
TENTER Y-AXIS SYMBOLS LOCATIONS VECTOR' 


mmnm Nnnn Nme mne mmmn wS rnana Nn Onn Amannan 


fed fad fd fd pd dd pd pd dp ә К ә (010 10 10 LO LO DQ Ш o III ЮО И WO I со со со == со 00 C300 
NN IFKNPINPKSPKNIPPPBbmóBIMBmBpmBÁÀyYxHOOOOOOOOOO:D0 -J0)U Е Пас СО А 2920 0123: ФФ Олд ЕО СТО СОЈ ОСЕ ОМ О 


САО О О0о УО» сл HF CO FO O3 O (00 00 У О сл FE ОУ МОДА LI =u suauu Oana ҚУА лл лл лл лл лл 


О О О ШШ ГЕШ ГЕНЕ | ИШ ГЕНЕ ГЕНЕ ГЕНЕ ГЕШЕ Г ШШ ТЕШЕ ШШ БИШ 2 Суз л JL JL. JL JL JL JL JL JL 
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> О 40 00 [2] ~ 09 О 4 ОУ КМ ЊУ ОЗ 00 00 ~Ј ОУ Сп 42 05 МА О (O 00 G3] -J 
СРЈ и пи или и ми аи и и и Су 


'OR ENTER 0 FOR SYMBOLS AT DEFAULT LOCATIONS OR NO 


'Il.E. 10 1 ОК УЕСТОВТТЛИВ КЕНШІ 
'"1_01 WILL SHOW LO AT Y= 1, MID AT Y=0, AI AT Y IS 


YO<N 
TTHESE AXIS CONTROL ENTRIES WILL МОН ВЕ ВОМ! 


RUN PAGEAX 

TDO YOU WANT TO RERUN THE PLOT INPUTS YOU AUD: 
'BEFORE RUNNING THIS AXIS CONTROL FUNCTION 

ENTER O FOR NO OR 1 FOR YES' 


CO<N 

-(COs1)/L6 

'DO YOU WANT TO DO ANOTHER AXIS CONTROL PAGE?! 
Е О FOR NO OR 1 FOR YES'! 


<N 
г ПӘЛЕ 
'DO YOU WANT TO RESTART THE FUNCTION?! 


ty 
л 


LE:'IF YOU DO NOT YOU WILL EXIT THIS FUNCTION! 


"ТЕ YOU EXIT THIS FUNCTION AND WANT TO RETAIN THIS 
'USE THE KEEP FUNCTION AND THEN YOU CAN RETURN TO 


'EY ENTERING )OFF HOLD' 
"ТЕ YOU WANT TO RETURN TO CMS, SIMPLY ENTER )OFF HOLD 


реши: 


"ENTER O FOR EXIT OR 1 FOR RESTART' 
<< 

EE 

l 
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The following file is the exec file, 


and runs the Split Linear Fit smoothing program. 


APPENDIX B 
SPLIT LINEAR FIT PROGRAM 


SPTLIN EXEC 


SPTLIN EXEC, which activates 


Chapter VI contains 


instructions on how to use this smoothing program. 


&TRACE 

SET BLIP ж 

GLOBAL  TXTLIB  VLNKMLIB VALTLIB  VFORTLIB IMSLSP  NONIMSL 
CMSLIB 

CLRSCRN 

&TYPE YOU HAVE INITIATED AN ALGORITHM 

&TYPE TO SMOOTH A SET OF DATA USING 

&TYPE ' SMOOTHING WITH SPLIT ЛЫНАН ЕТТІ 

&TYPE DEVELOPED BY MCDONALD AND OWEN OF 

ОРЕ STANFORD UNIVERSITY DEPT. OF STATISTICS 

- STRT &TYPE w**kw*kw*k kk kk kokokokok ook oo ook oc c oo Go ooo oc 0 o GG o 
&TYPE IN ORDER TO USE THIS ALGORITHM USE A 2M МАСНІМЕх х 
TYPE kK KKK KKK ROO OOOO OOOO OOOO Ж 
«ТҮРЕ 

&TYPE 

&TYPE DO YOU WISH TO CONTINUE? 

&TYPE ENTER У FOR YES OR ANY OTHER KEY TO EXIT: 

&READ VAR &CONT 

CLRSCRN 

&IF &CONT NE Y &COTO -EXIT 

&TYPE IN ORDER TO USE THIS ALGORITHM YOU MUST 

pos HAVE ON HAND THE FOLLOWING: 

“ТҮРЕ 1. FILENAME OF DATA FILE (FILETYPE DATA) 

а WITH DATA TO BE SMOOTHED. 

&TYPE 2. IF DATA POINTS ARE NOT IN CHRONOLOGICAL ORDER, 
&TYPE YOU NEED TO HAVE A FILE (FILETYPE ORDER) 
«ТҮРЕ WITH INDICES CORRESPONDING TO DATA POINTS 
oe INDICATING THE ORDER OF TRE DATA POINTS. 
СЕГРЕ З. FILENAME OF DATA FILE WHERE SMOOTHED OUTPUT 
&TYPE WILL BE WRITTEN OR IF YOU WANT 

&TYPE TO WRITE OUTPUT INTO APL HAVE ON HAND 

&TYPE THE APL VARIABLE AND WORKSPACE NAMES 

ВЕ ПЕ iG SORE THE OUTPUT.: 

«ТҮРЕ Ч. TRE NUMBER OF WINDOW SIZES 

«ТҮРЕ AND THE VALUE OF THE WINDOW SIZES 

&TYPE THAT YOU WANT TO ATTEMPT 

p ON THE SMOOTHINC OF THE DATA. 

САРЕ 5, THE MINIMUM WINDOW SIZE THAT 

‚ЕЁ CAN ER ATTEMPTED BY THE AGCORITHM. 

&TYPE DO YOU WISH TO CONTINUE? 

&TYPE ENTER Y FOR YES OR ANY OTHER KEY TO EXIT: 

&READ VAR &CONT 

CLRSCEREN 


&IF &CONT NE Y &COTO 


&TYPE 


- EXIT 
CAN YOU ACCESS 2M OF STORACE ON THIS DISK? 
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&TYPE ENTER ME YES ОН № ВОН ВО. 
«ТЕ ад x N &GCOTO Е 


СН 
&TYPE CONTAINS THE DATA TO BEA SMOOTHED: 
&READ ARGS 
«ІР &N = 0 &COTO -TELL 
&IF &N » 1 &cOTO -TELL 
STATE &1 DATA A1 
&IF &RC NE O &GOTO - ERROR 


CLRSCRN 
-ORDR CLRSCRN 
Se ARE DATA POINTS TO BE SMOOTHED 
ee IN снос ee ORDER 
E ENTER Rd IESU FOR NO: 
SREAD VAR «с 
&IF &CONT EQ Y 7 &GOTO zc 
&TYPE THE DATA POINTS ARE NOT 
&TYPE IN CHRONOLOGICAL ORDER? 
&TYPE ENTER FILENAME OF FILE (FILETYPE ORDER) 
&IYPE THAT CONTAINS ORDER INDICES 
&READ VAR &ORD 
STATE &ORD ORDER A1 
&IF &RC NE 0 &COTO -ERROR 
- со Со би 
«ТҮРЕ 


HE DATA YOU WANT TO SMOOTH 
IN &1 DATA 


5 
STYPE WHERE DO YOU WANT TO WRITE THE SMOOTHED OUTPUT? 
&TYPE CMS OR APL 
&TYPE OR YOU CAN PLOT THE SMOOTHED OUTPUT 
&TYPE SINCE YOU ARE LOGGED ON À 
&TYPE TERMINAL THAT CAN ACCESS GRAFSTAT, 
&TYPE BUT THE QUTPUT MUST BE STOR 
&TYPE IN AN APL VARIABLE. 
&TYPE ENTER APL OR CMS 
&READ VARS &PLA 
&IF &PLA EQ APL &GOTO -AP1 
&TYPE 
CLRSCRN 
&TYPE THE SMOOTHED QUTPUT WILL BE WRITTEN 
&TYPE INTO A CMS FILE (FILETYPE DATA) 
&TYPE ENTER ONLY THE FILENAME. 
&READ VAR &FN 
&TYPE THE SMOOTHED OUTPUT WILL BE WRITTEN 
&TYPE INTO THE FILE &FN DATA 


9 б 
N 
на 
ч 
xj 
У ДЈ 


&GOTO - 

“АРА ӨТЕР, 

&TYPE ENTER THE NAME OF THE 

&TYPE APL VARIABLE TO HOLD THE OUTPUT 

&READ VAR &À 

&TYPE DO YOU WANT TO PLOTOLTE QUPPRUE 

&L YPE ENTER Y FOR YES OR N FORNO: 

&READ VAR &GRF 

&IF &CRF EQ Y &COTO -PLOT 

&TYPE ENTER THE NAME OF THE APL WORKSPACE 

&TYPE THAT WILL CONTAIN «А 

&READ VAR &WKS 

&TYPE IS &WKS AN EXISTING MORE OR A NEW WORKSPACE? 
«ТҮРЕ ENTER O FOR EXISTING OR N FOR NEW: 

&READ VAR &AGE 

-PLOT БЕМ = TE 

-СОМ «ТҮРЕ 

&TYPE PLEASE READ THE FOLLOWING INSTRUCTIONS VERY CAREFULLY 
&TYPE ARE YOU READY TO START THE SMOOTHING PROGRAM? 

&I YPE ENTER У FOR YES ANY ОРНЕЕ КЕ E OREK 

&RBAD VARS &O 

CLRSCRN 

&IF SO NE Y «6070 -ЕХІТ 

&TYPE PLEASE WAIT THE SMOOTHING PROGRAM IS BEING COMPILED 
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FI OM CLEAR 
FI 05 CLEAR 
FI 06 CLEAR 
Beem? CLEAR 
FI 08 CLEAR 
I O9 


F CLEAR 

FORTVS SPTEIN (LVL Sep SDUMP 

FIL Ou" DISK &ORD ORD 

FIL 07 DISK &1 DATA 

EL sy DISK &FN DATA (RECFM FBA LRECL 80 BLKSIZE 800) 
&TYPE PLEASE WAIT SMOOTHING PROGRAM IS BEING LOADED 
LOAD SPTLIN (START 

CLRSCRN 

ERASE SPTLIN LISTING 

Poe SPTLIN TEXT 

ERASE LOAD MAP 


&IF &PLA EQ CMS &GOTO -EX 
«ТЕ &CRF EQ N &COTO -NCRF 
CP TERMINAL APL ON 

&STACK )LOAD SPTLIN 
&STACK &A <CMSRE 

&STACK &FN 

&STACK DATA 

&STACK М 

&STACK «А < ,&A 

&STACK сл <CMSREAD 

&STACK &1 

&STACK DATA 

&STACK М 

&STACK &1 <+ ,&1 

&STACK )SAVE' 

&TYPE ****xPLEASE WAIT, LINKING TO CRAFSTAT'*************x* 


&STACK )LOAD GRAFSTAT 
&STACK DUM <CMS 'CLRSCRN! 
&5ТАСК е с-н SPTLIN 
&5ТАСК 5Т ЕТ 


О ПАР 
- МСКЕ CP TERMINAL APL ON 
&STACK )LOAD SPTLIN 
&STACK &A «<CMSKEAD 
&STACK «ЕМ 
&STACK DATA 


«ТЕ SAGE EQ O &STACK deum &WKS 
N &STACK )WSID &WKS 
&STACK )PCOPY SPTLIN «А 
SAVE 
OFF HOLD 


AEC APL 

-DRP ERASE &FN DATA x 

CP TERMINAL APL ON 

&STACK )JLOAD SPTLIN 

&STACK JERASE &A 

&STACK JERASE &1 

&STACK SAVE 

&STACK JOFF HOLD 

EXEC APL 

ШЕЛ STIPE YOU HAVE FINISHED. 

&EXIT 1000 

-TELL &TYPE YOU HAVE ENTERED TOO MANY OR 
«ТУРЕ NOT ENOUGH ENTRIES ABOUT DATA FILE 
&TYPE YOU NEED TO BEGIN AGAIN 

ЕЛҮРЕ Dr dts SPTLIN 

EREXIT 10 

TERROR STYPE ABOVE ENTERED FILE DATA 
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&TYPE DOES NOT EXIST ON YOUR A-DISK 
ee CHECK рт AND THEN BEGIN AGAIN BY ENTERING 


101 
-EXIT &TYPE YOU HAVE FORCED AN EXIT ON THIS SMOOTHING EXEC 
&ТҮРЕ IF YOU До BEGIN AGAIN ENTER 


&EXIT ТО? 
2. SPTLIN FORTRAN 


The following file is SPTLIN FORTRAN which does the actual 
smoothing of a data set. The subroutines used in this program were 
developed by McDonald and Owen [Ref. 10] as stated in Chapter I. 
These subroutines were originally written in the C computer language. 
The author of this thesis translated the C language subroutines and 
combined them into an interactive FORTRAN program which follows. 


DOUBLE PRECISION WEIGHT, Y(1000),X(1000),W( 1000), TRY(1000,8,9) 

DOUBLE PRECISION TSMO(1000),SMOOTH(1000) 

REAL INFIN, CEPS, MISVAL, RESCAL, WTPOW 

INTEGER CMXOBS, CMXTRY, MNWNSZ, BASE, TRYSPN(10), NOBS, NTRYS, 
*NOTMIS 

COMMON /CONSTS/ INFIN,CEPS, MISVAL, RESCAL,WTPOW 

WRITE(5,2) 


2  FORMAT(1X, ENTER THE NUMBER OF DATA POINTS TO BE SMOOTHED 
*—|NTEGER VALUE:' /) 
READ(6,*)NOBS 
13 DO 151=1,NOBS 
15 Wil) = 1. 
14 WRITE(5, 16) 
16 FORMAT(1X,’ARE THE INPUT DATA POINTS IN CHRONOLOGICAL ORDER?” 
* /,1X,’ENTER 0 FOR NO OR 1 FOR YES’,/) 
READ(6,*) ODR 
IF(ODR.EQ.1)GO TO 17 
READ(4,*)(X(I),l=1,NOBS) 
GO Tos 


17 DO 19 1=1,NOBS 

19 X(I) = FLOAT(I) 

18 CALL FRTCMS(’CLRSCRN ’) 
WRITE(5,5) 


5 — FORMAT(1X, ENTER THE NUMBER WINDOWS TO BE USED 
*-—--INTEGER VALUE ’) 
READ(6,*)NTRYS 
CALL FRTCMS(’CLRSCRN ’) 
WRITE(5, 10) 
10 FORMAT(1X,’NEXT ENTER THE WINDOW SIZES IN INCREASING ORDER 
*..—~---INTEGER VALUES’,/) 
DO 8 1=1,NTRYS 
WRITE(5,9)I 
9 FORMAT(1X,’ENTER WINDOW SIZE NUMBER’,I4,/) 
READ(6,*)TRYSPN(I) 
CALL FRTCMS(’CLRSCRN ’) 
8 CONTINUE 
WRITE(5, 11) 
11 FORMAT(1X,’ENTER VALUE OF THE MINIMUM WINDOW SIZE 
*_-INTEGER’,/) 
READ(6,*)MNWNSZ 
CALL FRTCMS(’CLRSCRN ’) 
WRITE(5,20) 
20 FORMAT(1X,*****PLEASE WAIT PROGRAM NOW RUNNING********/) 
RESCAL = 1.0 
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WTPOW = 2.0 
READ(7,*)(Y(I),l = 1, NOBS) 
IF(ODR.EQ.1)GO TO 21 
CALL SORTER(X,W,Y,N) 

21 CONTINUE 
DO 121=1,NTRYS 
DO 12 J=1 NOBS - 


TRY(J,l, 
12 CONTINUE 
0061 « 1, NTRYS 
CALL RUNLRC(NOBS, X, Y, W, NTRYS, MNWNSZ, TRYSPN,I,TRY) 
6 CONTINUE 
CALL COMPWT(NOBS, NTRYS, TRY) 
CALL COMTRY(NOBS, NTRYS, TSMO, TRY) 
0071 = 1, МТВУ5 
CALL RUNLRC(NOBS, X, TSMO, W, NTRYS,MNWNSZ, TRYSPN,I, TRY) 
7 CONTINUE 
CALL COMPWT(NOBS, NTRYS,TRY) 
CALL COMTRY(NOBS, NTRYS, SMOOTH, TRY) 
WRITE(8,4) (SMOOTH(I)I = 1, NOBS) 
4 — FORMAT(2X,5(F12.6,2X)) 
STOP 
END 


EMEN ГТ ane A AER AKAARAARAARARKARRARARARRARK 
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SUBROUTINE COMPWT(NOBS, NTRYS,TRY) 
DOUBLE PRECISION LAMBDA, TEMP,MIN, TRY(NOBS,NTRYS,9) 
REAL INFIN,CEPS, MISVAL, RESCAL,WTPOW 
INTEGER NOBS, NTRYS, NTMS, A 
COMMON /CONSTS/ INFIN,CEPS, MISVAL,RESCAL,WTPOW 

DO 1J = 1, NOBS 
MIN = INFIN 
LAMBDA = 0. 
NTMS =0. 
DO 21 = 1, NTRYS 
TEMP = TRY(J,I,6) 
A = NOTMIS(TEMP, MISVAL) 
IF(A.EQ.1)GO TO 3 
GO TOG 
3 NTMS = NTMS + 1 
LAMBDA = LAMBDA + TEMP 
IF(TEMP.LT.MIN)MIN =TEMP 
4 TEMP = TRY(J,1,4) 
A = NOTMIS(TEMP,MISVAL) 
IF(A.EQ.1)GO TO 5 
GO TO 6 


5 NTMS = NTMS + 1 
LAMBDA = LAMBDA + TEMP 
IF(TEMP.LT.MIN)MIN — TEMP 
6 TEMP = TRY(J,1,5) 
A = NOTMIS(TEMP,MISVAL) 
IF(A.EQ.1)GO TO 7 
GO TO2 
7 NTMS = NTMS + 1 
LAMBDA = LAMBDA + TEMP 
IF(TEMP.LT.MIN)MIN — TEMP 
2 CONTINUE 
LAMBDA = LAMBDA/NTMS 
LAMBDA = LAMBDA - MIN 
IF(LAMBDA.GT.0.)LAMBDA = 1./(LAMBDA*RESCAL) 
IF(MIN.LE.0.)MIN = CEPS 
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DO 81 = 1, NTRYS 
TRY(J,1,9) = WEIGHT(TRY(J,I,6), MIN, LAMBDA,J,1) 
TRY(J,,7) » WEIGHT(TRY(J,l,4), MIN, LAMBDA, J,l) 
TRY(J,,8) - WEIGHT(TRY(J,l,5), MIN, LAMBDA, J,l) 

8 CONTINUE 

CONTINUE 
RETURN 
END 


Стек АТА ОБА КАЛЕ 


d. 


Слики нае 


SUBROUTINE COMTRY(NOBS, NTRYS, SMOOTH, TRY) 
DOUBLE PRECISION RSUM,WSUM,T,SMOOTH(NOBS), TRY(NOBS, NTRYS,9) 
REAL CEPS,MISVAL 
INTEGER NOBS, NTRYS, A 
COMMON /CONSTS/ INFIN,CEPS, MISVAL,RESCAL,WTPOW 

DO 1J = 1, NOBS 
RSUM = 0. 
WSUM = 0. 
DO 21 = 1, NTRYS 
T = TRY(J,I,3) 
A = NOTMIS(T,MISVAL) 
IF(A.EQ.1)THEN 
RSUM = RSUM 4 TRY(J,,9)* TRY(J,l,3) 
WSUM = WSUM + TRY(J,1,9) 
ELSE 
COTOS 
ENDIF 
3 T = TRY(J,1,1) 
A = NOTMIS(T,MISVAL) 
IF(A.EQ.1)THEN 
RSUM = RSUM + TRY(J,1,7)*TRY(J,1,1) 
WSUM = WSUM + TRY(J,I,7) 
ELSE 
со то 4 
ENDIF 
4 Т = TRY(J,1,2) 
A = NOTMIS(T,MISVAL) 
IF(A.EQ.1)THEN 
RSUM = RSUM + TRY(J,I,8)*TRY(J,I,2) 
WSUM = WSUM + TRY(J,I,8) 
ELSE 
GOTO 2 
ENDIF 
2 CONTINUE 
IF(WSUM.GE.CEPS)THEN 
SMOOTH(J) = RSUM/WSUM 


ELSE 
SMOOTH(J) = MISVAL 
ENDIF 
1 CONTINUE 
RETURN 
END 


ымыны ыы ыы ыы ыы ыы ыы 


С 
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SUBROUTINE RUNLRC(NOBS, X, Y, W, NTRYS, MNWNSZ, TRYSPN, I, TRY) 
DOUBLE PRECISION AT,BT, MEANRQ, YMEAN, Y2MEAN, 
* — SLOPEINTER,X(NOBS), Y NOBS),W(NOBS) 
DOUBLE PRECISION XSUM, YSUM,X2SUM, Y2SUM,XYSUM, 
*  WSUM,XVAR,XMEAN,X2MEAN,XYMEAN,EPS, TRY(NOBS, NTRYS,9) 
REAL CEPS, MISVAL 
INTEGER NOBS, NTRYS, JL, JR, TRYSPN(NTRYS), MNWNSZ, 
* — RFLAG,CFLAG,LFLAG, I, JC 
COMMON /CONSTS/ INFIN,CEPS, MISVAL, RESCAL,WTPOW 
JL = NOBS/4 
JR = 3*JL 
EPS = X(JR) - X(JL) 
2 IF (EPS.LE.0.0.AND.JR.LT.NOBS)THEN 
IF(JR.LT.NOBSJJR = JR + 1 
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11 
12 


13 
14 


15 


16 


IF(JLGT.1)JL — JL - 1 
EPS = X(JR) - X(JL) 
ELSE 
GOTO 1 
ENDIF 
СОТО? 
CONTINUE 
EPS = EPS*CEPS 
EPS = EPS**2 
IF(TRYSPN(I).LT. MNWNSZ)TRYSPN(I) = MNWNSZ 
XSUM = 0. 


YSUM = 0. 
WSUM = 0. 
X2SUM = 0. 
Y2SUM = 0. 
XYSUM = 0. 
KI = MNWNSZ - 1 
00418 = 1, Kl 
CALL UPDATE(1, X(JR), Y(JR), , WOR), XSUM, YSUM, 
WSUM,X2SUM,Y2SUM, XYSUM) 
TRY(JR,I,1) = MISVAL 
TRY(JR,I,4) = MISVAL 
JL = JR-TRYSPN(I) + 1 
JC = JR - TRYSPN(I)/2 
KT = NOBS - MNWNSZ + 1 
IF(JL.GE.1)GO TO 7 
LFLAG = 0 
GO TO 8 
LFLAG = 1 
IF(JC.GE.1.AND.JC.LE.NOBS)GO TO 9 
CFLAG - 0 
GO TO 10 
CFLAG - 1 
IF(JR.LE.NOBS)GO TO 11 
RFLAG — 0 
EO TO 12 
RFLAG = 1 
IF(RFLAG.EQ.1)GO TO 13 
GO TO 14 
CALL UPDATE(1, X(JR), Y(JR), WR), XSUM, YSUM, 
WSUM,X2SUM,Y2SUM, XYSUM) 
XMEAN = XSUM/WSUM 
X2MEAN = X2SUM/WSUM 
YMEAN = YSUM/WSUM 
XYMEAN = XYSUM/WSUM 
Y2MEAN = Y2SUM/WSUM 
XVAR = X2MEAN - XMEAN**2 
IF(XVAR.LE.EPS)THEN 
SLOPE = 0. 
ELSE 


SLOPE = (XYMEAN - XMEAN*YMEAN)/XVAR 
ENDIF 


INTER = YMEAN - SLOPE*XMEAN 
MEANRQ = Y2MEAN -(2.*INTER*YMEAN)- 
(2.“°SLOPE*XYMEAN) + (INTER**2) + 
(2."INTER^SLOPE*XMEAN) -- (X2MEAN^SLOPE**2) 
IF(LFLAG.EQ.1)GO TO 15 

GO TO 16 

AT =TRY(JLI,3) 

BT =TRY(JL,I,6) 

CALL EVALFT(INTER,SLOPE, MEANRG,JL,W,X,Y,AT,BT, |, WSUM) 
TRY(JL,I,3) =AT 

TRY (JL,I,6) = BT 

IF(RFLAG.EQ.1)GO TO 17 

GO TO 18 

AT =TRY(JR,I,1) 

BT =TRY(JR,I,4) 

CALL EVALFT(INTER,SLOPE,MEANRQ,JR,W,X,Y,AT,BT,], WSUM) 
TRY(JR,I,1) 2 AT 

TRY(JR,I,4) = BT 
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18 IF(CFLAG.EQ.1)GO TO 19 
GO TO 20 
19 AT=TRY(JC,I,2) 
BT=TRY(JC,I,5) 
CALL EVALFT(INTER,SLOPE, MEANRQ,JC, W, X, Y, AT, BT, I, WSUM) 
TRY(JC,l,2) - AT 
TRY(JC,I,5) + ВТ 
20 IF(LFLAG.EQ.1)GO TO 21 
GO TO 22 
21 CALL UPDATE(0,X(JL), Y(JL), WU). XSUM,YSUM, 
* WSUM,X2SUM,Y2SUM,XYSUM) 
22 JR=JR+1 
JL = JL + 1 
JC = JC + 1 
IF(JLLE.KT)GO TO 6 
KL = JL 
DO 23 JL = KL, NOBS 
TRY(JL,,3) = MISVAL 
23 TRY(JL,1,6) = MISVAL 
RETURN 
END 


яя АЗ ae ee ee 
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INTEGER FUNCTION NOTMIS(AO, MISVAL) 

DOUBLE PRECISION AO 

REAL MISVAL 
IF(AO.GT.MISVAL)GO TO 1 
NOTMIS = 0 

RETURN 

1 NOTMIS = 1 
RETURN 
END 


Фе о ово со ооо роса тя с осо eee eee ee ааа 
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REAL FUNCTION WEIGHT(R,INTER2,SLOPE2,J, 1) 
DOUBLE PRECISION R,TEMP2,INTER2,SLOPE2 
REAL WTPOW,MISVAL 
INTEGER A.J 
COMMON /CONSTS/ INFIN,CEPS, MISVAL,RESCAL,WTPOW 
A = NOTMIS(R,MISVAL) 
IF(A.EQ.1)THEN 
TEMP2 = SLOPE2*(R - INTER2) 
IF(TEMP2.LE.0.)THEN 
WEIGHT = 1. 
ELSE 
IF(TEMP2.LT.1.)THEN 
WEIGHT = ((1.-TEMP2)**(INT(WTPOW)))*( 


: (1.-TEMP2)**(WTPOW- INT(WTPOW))) 
ELSE 
WEIGHT = 0. 
ENDIF 
ENDIF 
ELSE 
WEIGHT = 0 
ENDIF 
RETURN 
END 


CCA AKAARRRKRRRARERAARRRRRERARRRERARARERE RERRAERARARARARRR 


С 
CRA RAAARKARARRRARREREEE RRR RAERARRRRARRARRARARARRRRRRRRERL 
SUBROUTINE UPDATE(OP,A1,B1,C1,XSUM,YSUM,WSUM,X2SUM, 
*Y2SUM,XYSUM) 
DOUBLE PRECISION XSUM,YSUM,WSUM,X2SUM,Y2SUM,XYSUM,A1,B1,C1 
INTEGER OP 
IF(OP.EQ.0)THEN 
XSUM = XSUM - C1*A1 
YSUM = YSUM - C1*B1 
WSUM = WSUM - C1 
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X2SUM = X2SUM - C1*A1**2 
Y2SUM = Y2SUM - C1*B1**2 
XYSUM = XYSUM - C1*A1*B1 
ELSE 
XSUM = XSUM + СТ'А1 
YSUM = YSUM + C1*B1 
WSUM = WSUM + C1 
X2SUM = X2SUM + C1*A1**2 
Y2SUM = Y2SUM + C1*B1**2 
XYSUM = XYSUM + C1*A1*B1 
ENDIF 
RETURN 
END 


ПНЕ А АМА А А А RRAARRRARAARARARARARR 


РЕМЕТ 8 Se RARARARRARAARAARAARARARAAARAA ARE 


SUBROUTINE EVALFT(A,B, R2,JI,W,X,Y, TRYS, TRYR,|, WSUM) 
DOUBLE PRECISION TRYS, TRYR,WSUM,A,B,R2,W(JI),X(J1), 
*Y(JI), FIT, RES 
INTEGER JI, 
FIT — A 4- B*X(Jl) 
RES = Y(Jl) - FIT 


TRYS - FIT 
TRYR = (WSUM*R2 - W(JI)*RES*RES)/(WSUM - W(JI)) 
RETURN 


END 


MEE ЗА REAR ERA AAALSERAARRRAEARAAARARAR ARK 


РЕКЕ SK RARRAARRARAAAARRARAAAKARRRR ARARRARKA 


SUBROUTINE SORTER(X,W.Y,N) 
DOUBLE PRECISION X(N),W(N), Y(N),D(5000) 
INTEGER N,KEY(5000) 

DO 51-1,N 

5 КЕҮ(1)=1 

CALL SHSORT(X,KEY,N) 
DO 11=1,N 
1 DiI) = Wil) 
DO 21=1,N 
Ј= КЕҮ(І) 
2 W(l) - D(J) 
DO 31=1,N 
3 D(I) - Y(I) 
DO 41=1,N 


Y(l) = D(J) 
RETURN 
END 


БЕТ = = 88 RASA ARAARARAARARAARARRARARARAA ARRAS 
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BLOCK DATA 

COMMON /CONSTS/ INFIN,CEPS, MISVAL, RESCAL,WTPOW 

REAL INFIN,CEPS,MISVAL, RESCAL,WTPOW | 

DATA INFIN,CEPS,MISVAL, RESCAL,WT POW /1.0E30,1.0E-10, 
*-1.0E30,1.0,2.0/ 

END 


3. — SPTLIN VSAPLWS 


The following two APL functions are used in conjunction with the 
two files listed above. They were developed by the author of this 
thesis in order to plot the smoothed data/results. They are the main 
APL functions within the APL workspace SPTLIN. The first APL 
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function links the user of the smoothing program with GRAFSTAT and 
gives a user familiar with GRAFSTAT the opportunity to proceed into 
GRAFSTAT where a greater variety of graphic functions are available. 


МГ АЕ 

DUM-CMS 'CLRSCRN! 

"THE ENTIRE DATA FILE THAT YOU WANTED SMOOTHED HAS 
TRANSFERRED! 

'TO THIS WORKSPACE SO THAT YOU MAY BE ABLE TO PLOT 


'THE SMOOTHED AND UNSMOOTHED DATA. 
THE UNSMOOTHED DATA IS IN THE VARIABLE WITH THE SAME 


'THE DATA FILE THAT YOU HAVE YOUR INPUT DATA IN.!' 


1 
'DO YOU WISH TO GO INTO APL OR CONTINUE?! 
LENTER 0 FOR APL OR 1 FOR CONTINUE! 
<- 
+>13+Сх8 
'YOU WILL BE SENT TO APL AFTER YOU HAVE READ THIS 
RTANT TEXT. 
pyt“ AFTER YOU HAVE FINISHED WORKING IN APL AND WISH TO 
ЕТ Зар РГОТЕВхххххжхх«МОТІСЕ THAT PLOTER HAS ONLY ONE 


' NOW ENTER O AGAIN' 
Сеш 


ее 
m= 


NN PRP -p|PpDPammmpmpbtu)po 0o 42 О о а] 
Dy 


L JL JL JL JL н) LILI Ltt 


>C 
PLOTER 


DTPOWOND NOL WONPOWMLUILILI Ss 


The next APL function creates the APL variables to used in the APL 
'PLOT' screen. This plotting option is made available to the user 
through the above APL function. The user can use this APL function 
to do the plotting or the GRAFSTAT graphics functions. A user need 
not fully understand how to use the GRAFSTAT plot screen in order to 
use this function. Several examples are shown with each of the queries 


so that the user can see what the entry should look like. 


ОТЕБ; ;СО:РИМ:Р: БУ; ТТ; ТГ;ТР; ХЕ; ХО; Х5: ХТ; ХУ: ХУ; УГ; УО: УЗ; УТ; УИ 
1 ромесмз ТСЪВЗСЕЛ! 

] TYOU"RAVE ACTIVATED THE PLOTTING FUNCTION' 

1 'IT IS ASSUMED THAT THE USER IS FAMILIAR WITH THE 
AFSTAT PLOT FUNC. 

] TAND THE AXIS CONTROL FUNCTION! 

ит YOU RECEIVE (MAKE) AN ERROR MESSAGE DO THE 

1 а ENSURE THAT VM READ IS DISPLAYED IN LOWER RIGHT 

RNER OF SCRE 


га сл боје Сус гг С гг 
POR md 00 -C€350)€5 01 Еру мн (ч 


| 12: PRESS THE а 
1 '70 UNDERSCORE A LETTER HOLD THE APL/ALT KEY DOWN AND 
ESS THE LETTER 
D]. "SEE PLOPTING FUNCTION WILL RESTART AT THE BEGINNING! 
1 'THE PLOTTING FUNCTION САМ ВЕ EXITED AT ANY INPUT 
сй BY ENTERING! 

=> 
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'AT ANYTIME THAT YOU EXIT THE PLOTTING FUNCTION' 
'YOU WILL BE IN THE GRAFSTAT WORKSPACE'! 

ТЕ YOU WISH TO RETURN TO CMS ENTER' 

1 JOFF HOLD' 


1 
1 1 


LB:'ENTER X VARIABLE(S) (ENCLOSED IN QUOTES), ТЕ 
RING MORE THAN ONE VARIABLE! 
| SEPARATE VARIABLES WITH SEMICOLON AND USE QUOTES! 
ЕСИТУ ORV WIXI: xX?! 
XV«N 
DUM<CMS 'CLRSCRN 

Y VESFABLECS) (ENCLOSED IN QUOTES AND MUST BE 
AME LENGTH AS X) 


UE ENTERING MORE THAN ONE VARIABLE, SEPARATE WITH 
'AND REMEMBER TO USE QUOTES ENCLOSING ENTIRE STRING! 
'E Ge prey tt OR Шау 22171 1 


FWNRONWONOM FW 


YV 
TENTER A VECTOR VOCAL ING TYPE(S) OF PLOT: б=5уМ 
ОЕ ОО Л ОК ОЛ ОЛО О ОК 1 0 OR 1 1' 


ТР<М 

Бар) 2007/01 

'ENTER TYPE OF SYMBOL CORRESPONDING TO EACH SYMBOLS 
PLOT (IN QUOTES) 


En С. pee aK YOU.CAM USE .**x! 


SUE /2P )20)/ LP 
Е eo INDICATING TYPE(S) OF LINES; 1=SOLID 
| 275 1 OR 3 OR 1 3 OR ANY OTHER COMBINATION OHR LINE 


Ню SWWWWSNNYNYNEAN ММММ E. E 
Саси мени аи ка лл лак ууш л щл реси JL JL IL JL It d 
* C 
О 


MEWNORPOWWOSNOMF о О но 00 4 бода сл 


:71<1 
Е’ 


Ы = DUM<CUS Mono HII 
‘ENTER SCALE OF X-AXIS (IN QUOTES) OR P (IN QUOTES) 
PREVIOUS | SCALE 

Rc OW ОСАМ АМАХ ОҚ ТІРІ 1 


5<М 
ТЕМТЕВ SCALE OF Y-AXIS (IN QUOTES) OR P (IN QUOTES) 
PREVIOUS CALE 
ДО» ЕО А ТЕАИ ИИО МАХИ OF TIP! ! 


1 

1 <N 

] TENTER ‚ТНЕ PLOT HEADER (IN QUOTES) OR EMPTY QUOTES! 

] „С. "TITLE" OR 

1 D 

] DUM<CMS 'CLRSCRN' 

] TENTER X-AXIS LABEL (IN QUOTES) OR ! 

2 E PAIR OF EMPTY QUOTES FOR NO LABEL OR TO USE AXIS 


\E.G. ''LABEL'' OR! U | 


1 

1 

] TENTER Y-AXIS LABEL (IN QUOTES) OR' 

2 n PAIR OF EMPTY QUOTES FOR NO LABEL OR TO USE AXIS 


EC EU UEABDEUVULQR T тү: 


] 
1 YL< 

] TDO YOU WANT TO RUN THIS PAGE?! 
] 'ENTER O FOR NO OR 1 FOR YES! 

| CO«N 

| DUM<CMS 'CLRSCRN' 

1 E CO=0) 

1 Ae Пе 

1 

1 


аааз Су гааг агага гагат гага гт ҹаг Огги а Onnan 


б» Оз» О» О» О» О» Оз Оз бз Оз Сул ел‹елелСу‹ ал ‹ал‹л‹ел‹л‹л.Е.Е ЕСУ Е.Е.ЕСУКЕ.Е.ЕЕ I CO F4 0D 


| PLEASE WALT RUNNING PAGE! 
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RUN PAGESAM 
TDO YOU WANT ТӨСЕХІТ дал. FUNCTION? ' 
с O FOR NO OR 1 FOR YES' 


<N 
aoa 1)/LE 
"рО YOU WANT TO RESTART THIS FUNCTION?! 
"ЕМТЕВ 0 FOR NO OR 1 FOR YES! 


CO<N 
DUM<CMS 'CLRSCRN' 
(СОВЕ 


'THE ONLY THING LEFT TO DO IS THE AXIS CONTROL! 
TRÜCTING' THE PARTIAL PLOT THAT YOU HAVE JUST FINISHED 

'ENTER A 3 ELEMENT VECTOR FOR PARTIAL PLOT' 

„157 ELEMENT, 1(0): LINES AND SYMBOLS ARE (NOT) SHOWN 


EEN 
‘OND ELEMENT, 1(0): HEADER AND AXES ARE (NOT) SHOWN ON 


Q 
ХЈ 


ЕМ! 

) SHOWN ELEMENT, 1(0): AXES, CRIDS, AND CRID LINES ARE 
p 110 WILL SHOW EVERYTHING ON CRAPH EXCEPT AXES 

CRID ENES 


TENTER A 4 ELEMENT E FOR AXES Ди GRID C 
'157 ELEMENT, X-AXIS: 0 EXORTU ED JOE ОА AT 


'2ND ELEMENT; Y AXIS l LEFT; 3 = RICHT, ОЛТИ fee 


L3 RD ELEMENT, VERTICAL GRID LINES: O=NO GRID, 
гаа Ди 22590 BENED 
ELEMENT, HORIZON. GRID LINES: O=NO Chis 
ТТЕР, “OR 2=SOLID 
'E.G. 2 1 2 2 WILL DISPLAY AXIS AT TOP AND LEFT AND 
D GRID LINES' 


<- 
L8:'PLEASE WAIT RUNNING PAGE! 


ESAM 
LÀ:DUM-CMS 'CLRSCR 
СЕДЕА AXIS TIC "MARKS LOCATION VECTOR! 
'OR ENTER O FOR STANDARD TIC MARKS' 
'OR ENTER 1 FOR NO_TIC MARKS 
ПА C. 1 Orme РО ome Or NAME OR OORE 
Lt 2 11 WILL SHOW TIC MARKS AT x=1, X=5, AND И 
<- 
TENTER X-AXIS SYMBOLS (IN QUOTES)! 
"OR ENTERT O ATCP QUOTES FOR STANDARD SYMBOLS! 


'OR ENTER 1 WITHOUT QUOTES FOR NO SYMBOLS' 
„Е.С. 111970:1971'' О А ТЕСТОВ МАМЕ ОВ 0 ОВ 1' 
<- 
TENTER X-AXIS SYMBOLS LOCATIONS VECTOR! 
р, ОЁ ENTER O FOR SYMBOLS AT DEFAULT LOCATIONS OR NO 
'Е.С. 6 18 ОБ д VECTOR NAME OR _0' 
18 48 WILL SHOW 1970 AT X=6 AND 1971 AP Х=18' 


о ' CLRSCRN'! 

TENTER Y-AXIS TIC MARKS Е VECTOR' 
'OR ENTER O FOR STANDARD TIC MARKS' 

OR oa 1 FOR NO LIC MARKS! 

КЕР QO 1 OR A VECTOR NAME ОК 0 ОК 
e 1 WILL SHOW TIC MARKS AT Ys 1, 


TENTER Y-AXIS SYMBOLS (IN QUOTES)! 
'OR ENTER O WITHOUT 200 TES FOR ON ee 


1 


1 
УЕ d PEE IU 


'OR ENTER 1 WITHOUT QUOTES FOR NO SYMBOL 
г.с. "да МІР HI'' OR VECTOR WANE SCO EE 


Jie 
TENTER Y-AXIS SYMBOLS LOCATIONS VECTOR' 


аиа уыты ыты ы ыы ыы ыты гг Каган ғал Сут Сол _____>___ 
КӘ КӘ КӘ ве нА РА РА рава ра ра рз ра рз кз Кана ва рано в в ро р а р оО ОСО Ш О Ш £O II] CO НК Со со со => со => 00 Су СО => ОО СО С со оо 1-141-111-1131 4-4 
МММ МММ мн н ES p. E 3 3. jp. pA 222 COO CO OO OO OO O (0 00 10) 01 НЕ СЗО ОЈ У ОЈ O O O (000 139 Очпщ в CONO ZEE O(O 00 9100 01 омно 


бел COh2 F2 O (0 00 102 01 F2 COND ES UO O CO 00 100 O1 HZ CO NB FS CO LÀ. ái LL NLI QI Ot abs alts LO QL BG ын ллы млн Gye tts Ls LLL OL ar ara 


LLLJL JL JL JU JL 2U JL JU JL QU JL JU JL OU JL Cy JUL JU JL JL JUL JL QU JL JU JL 3 
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rro ға Сұ ағыта г 
НУ нары мүн н Ыы Суы ыыы ананын А РА РАНА РА Кака 
TNF FN FEWNEDWFEF EF EWWWWWWWWWWNNYZNY 
Оф 001-10) = Ол ом оосо УО сл = ом Оо осом 
CI EE Е 


Сч 
(а 


„ОВ ENTER O FOR SYMBOLS AT DEFAULT LOCATIONS OR NO 


В ТО 1 0 VECTOR NAME OR Oo! 
1 ПИН ВЕ ОНОН BORAT Yee. MID AT Y=0, HI AT Y=1' 


T THESE AXIS CONTROL ENTRIES WILL NOW BE RUN' 


RUN PAGEAX 
TDO YOU WANT TO RERUN THE PLOT INPUTS YOU ENTERED ' 
'BEFORE RUNNING THIS AXIS CONTROL FUNCTION? '! 
ENTER O FOR NO OR 1 FOR YES' 


Co<N 
>+(CO=1)/L6 
'DO YOU WANT TO DO ANOTHER AXIS CONTROL PAGE?! 
К УРА О FOR NO OR 1 FOR YES' 
ES 
-(COz1)/L8 
' DO 700 WANT TO RESTART THE FUNCTION?' 


OE: EROU T DOT NOT YOU WILLE EXIT THIS FUNCTION! 


'IF YOU EXIT THIS FUNCTION AND WANT TO RETAIN THIS 
'USE THE KEEP FUNCTION AND THEN YOU CAN RETURN TO 
'DY ENTERING )OFF HOLD' 


E YOU WANT TO RETURN TO CMS, SIMPLY ENTER )OFF HOLD 


I 

На ПОШАО ЛАНКЕ ОК 1 FOR RESTART ' 
< 

пс поща 

1 
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APPENDIX С 
APL FUNCTIONS 


1. RANDOM NUMBER GENERATOR 


The following GRAFSTAT [Ref. 12] APL function generates Normal 
random deviates and was used to produce the N(0,1) noise added to 
basic functions used in Chapter IV. 


V NE NORRANDEP:; S; L31 


C1] P< 

[21 2 (80 

(31 I< 

Cu] F10: 1л<2 UNIRAND 0.5 0.5 

[5] T-(2xT)-1 

[6] S< Pei} 2 +202) к2 

C7] >Ғ10х15>1 

[3] м аи 2х95)+5)%0.5) 
[9] аи T<It+1)< 


The folowing GRAFSTAT [Ref. 12] APL function generates uniform 
random numbers which аге used in the above APL function, 
NORRAND. 


V R-N UNIRAN 
[1] Re (BE1]- 21122 E +100000000 


2. EQUAL-WEIGHT MOVING AVERAGE SMOOTHER 


The folowing GRAFSTAT [Ref. 12] APL function is the Moving 
Average smoother used to generate the associated smooth plots in 
Chapter IV. 


У УМ MOVAV X 
C1] ТҰН МАУ X 


The folowing GRAFSTAT [Ref. 12] APL function computes weights 
corresponding to the data values within the neighborhood W and does 
the weighted averaging within W. These values are the smoothed 
values which are transferred to the above APL function. 


о 


ЕСІН! 23 и W)^v/ zppX 
ПАД И L> 1) (Leow )<14 АрерХ 
23/1 ПРЕ МОС Рен 

Cu] Dli DULIT- D 
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C5] U<+DoJ<0 
[6] U-U*W[J-J*1]xDpJ'"*X 
ВУ | -6xL»J 


3. COSINE-WEIGHTED MOVING AVERAGE SMOOTHER 


The folowing APL function computes the cosine weights for the 
cosine-weighted moving average [Ref. 14: p. 394] with window size R, 
i.e. length: 

V WCW В 
[1] а WEIGHTS FOR A COSINE-WEIGHTED MOVING AVERAGE OF 
LENGTH R 
C2] НА e aOR DERI 

The folowing APL function is the Cosine Weighted Moving Average 
algorithm. It is part of the time series APL workspace TSERIES devel- 
oped at the Naval Postgraduate School. 

V S<WW RUNSMOOTH X;L;W;N;M;LW;I;R; Ша IDX 


ІЗІМ A RUNNING SMOOTH OF X WITH WEIGHTS 

82 1] а W MUST BE ODD YECTOR THAT ADDS Әр” TO 

[3] WW HAS AS 1ST ELEMENT THE ADVANCE STEP L FOR THE 
SMOOTHING WINDOW 

I] RESULT IS 2-ROW MATRIX WITH SMOOTH VALUES (ROW 1) AND 
INDICES (ROW 2) 

Es] L<l /1,1+WW 

[6] Дем 

Е? 1 LW 

[8] ХО. 5XL 

[9] >((IDX2LWe IS +/W))/LO 

[3008 'WEIGHTS W IN RUNSMOOTH "NOT ODD OR DONT ADD UP TO 1. 
PROGR TERMINATED ' 

[11] > 

m] 505: 

[13] МерХ 

[14] S<10 

[15] ТХ<=10 

[16] Re 1+1LW 

[17] M<N+1-LW 

[18] Т<=1-1 

ШО £10: а аи 

[20] Bes T/WXXUI-TR 

ШЕШІ UDXerX.IRIDX 

[22] 2110 

E235] 220 

124. а RETURN SMOOTHED VALUES AND THEIR INDICES IN ROW 2 


CE 
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APPENDIX D 
DATA SETS 


This appendix contains in tabular form the data sets used in this 
thesis. The first table shows the Daily Sea-Surface Temperature in 
degrees Centigrade collected at Granite Canyon, just south of Point 
Sur, California. This data set used in Figures 1.1 and 1.2 which were 
illustrations of a scatterplot and a smooth curve through the scatter- 
plot. The other three tables contain the Test Data sets used in the 
evaluation of the Supersmoother and the Split Linear Fit smoothers, i.e. 
Test Set One, Test Set Two, and Test Set Three. 
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TABLE 19 


DAILY SEA-SURFACE TEMPERATURE IN DEGREE CENTIGRADE 
AT GRANITE CANYON, CALIFORNIA 


TEMP. DATE TEMP. DATE TEMP. DATE TEMP. DATE TEMP. DATE TEMP. DATE 


08.8 1060 088 1111 09.7 1162 10.6 1213 13.7 1264 13.2 1315 
08.5 1061 08.5 1112 09.9 1163 10.7 1214 14.0 1265 12.4 1316 
08.5 1062 09.3 1113 10.1 1164 107 1215 144 1266 125 1317 
09.0 1063 09.1 1114 09.8 1165 10.8 1216 14.0 1267 12.0 1318 
08.8 1064 09.0 1115 09.8 1166 10.9 1217 14.8 1268 11.4 1319 
08.3 1065 09.0 1116 09.5 1167 11.2 1218 14.0 1269 10.8 1320 
08.9 1066 09.6 1117 09.8 1168 5 12 1383.8 11270 113 1321 
09.2 1067 09.0 1118 09.8 1169 11.0 1220 131 1271 111 1322 
09.2 1068 090 1119 10.2 1170 о елите 1272 11391323 
08.9 1069 093 1120 10.5 1171 11.3 1222 12.8 1273 11.9 1324 
09.8 1070 09.5 1121 09.9 1172 10.8 1223 11.8 1274 11.9 1325 
09.8 1071 108 1122 09.9 1173 10.9 1224 116 1275 12.1 1326 
09.8 1072 10.8 1123 10.0 1174 110 1225 118 1276 117 1327 
09.9 1073 10.6 1124 09.8 1175 11.2 1226 11.8 1277 11.7 1328 
09.6 1074 10.4 1125 09.7 1176 11.8 1227 122 1278 11.5 1329 
09.5 1075 09.8 1126 10.2 1177 12.1 1228 11.9 1279 110 1330 
09.6 1076 09.8 1127 09.9 1178 11.5 1229 11.2 1280 11.5 1331 
09.3 1077 103 1128 10.0 1179 11.7 1230 11.3 1281 11.5 1332 
09.8 1078 10.8 1129 10.0 1180 11.9 123111.5 1282 1191333 
10.4 1079 10.9 1130 09.9 1181 12.3 1232 11.0 1283 11.2 1334 
10.9 1080 10.0 1131 09.9 1182 12.5 1233 10.5 1284 10.9 1335 
10.8 1081 100 1132 09.8 1183 12.8 1234 10.4 1285 11.2 1336 
10.0 1082 10.0 1133 09.8 1184 11.8 1235 11.0 1286 10.8 1337 
10.0 1083 09.7 1134 09.9 1185 11.9 1236 11.2 1287 10.9 1338 
10.0 1084 100 1135 09.5 1186 14.0 1237 11.1 1288 10.7 1339 
11.0 1085 09.6 1136 09.9 1187 13.9 1238 11.8 1289 10.0 1340 
10.3 1086 089 1137 09.9 1188 13.4 1239 126 1290 09.6 1341 
09.7 1087 089 1138 10.4 1189 13.0 1240 13.2 1291 09.2 1342 
09.9 1088 089 1139 09.9 1190 13.2 1241 140 1292 09.6 1343 
09.9 1089 09.2 1140 10.5 1191 12.0 1242 13.8 1293 09.7 1344 
08.8 1090 089 1141 10.2 1192 12.0 1243 12.2 1294 09.8 1345 
08.9 1091 09.2 1142 10.0 1193 12.0 1244 12.1 1295 09.4 1346 
09.1 1092 09.4 1143 09.2 1194 11.9 1245 11.8 1296 09.8 1347 
09.3 109: 10.44 1144 09.5 1195 10.6 1246 11.3 1297 09.8 1348 
09.0 1094 100 1145 09.8 1196 10.5 1247 11.3 1298 09.7 1349 
09.3 1095 09.8 1146 10.0 1197 12.6 1248 11.5 1299 09.8 1350 
09.6 1096 10.1 1147 11.2 1198 12.5 1249 10.7 1300 09.5 1351 
09.8 1097 10.2 1148 11.2 1199 11.1 1250 099 1301 100 1352 
09.8 1098 100 1149 11.7 1200 10.8 1251 09.5 1302 09.9 1353 
09.8 1099 09.3 1150 11.8 1201 11.0 1252 100 1303 09.9 1354 
10.0 1100 09.2 1151 11.2 1202 11.0 1253 10.8 1304 10.0 1355 
09.8 1101 089 1152 11.2 1203 10.3 1254 096 1305 106 1356 
09.0 11042 090 1153 11.2 1204 112 1255 10.3 1306 10.7 1357 
09.3 1103 09.2 1154 11.2 1205 11.7 1256 10,8 1307 11.0 1358 
10.9 1104 09.4 1155 10.9 1206 12.0 1257 11.0 1308 10.8 1359 
09.7 1105 09.2 1156 10.9 1207 15.0 1258 110 1309 10.5 1360 
09.0 1106 09.2 1157 10.9 1208 15.8 1259 11.5 1310 10.8 1361 
09.4 1107 09.4 1158 10.6 1209 14.5 1260 12.2 1311 10.7 1362 
09.0 1108 09.5 1159 10.4 1210 13.3 1261 11.2 1312 10.7 1363 
09.1 1109 09.5 1160 10.3 1211 14.0 1262 12.4 1313 10.7 1364 
09.2 1110 097 1161 11.4 1212 13.8 1263 11.8 1314 10.7 1365 


149 


Y 


-0.5021245 
-2.2706658 
1.2849535 
0.7868391 
0.2750802 
0.5916070 
-0.4165126 
2.0161424 
0.4190598 
2.1970718 
0.7040685 
1.3516733 
-0.9261715 
-0.1411653 
1.8459821 
0.0010230 
1.2573502 
0.3599704 
0.6244237 
-0.5493385 
-0.4304499 
1.8645709 
0.8751194 
0.1610555 
0.2348275 
1.9017348 
1.0237749 
1.6334782 
1.5566808 
1.9562190 
1.6404860 
-0.0613807 
1.6950412 
2.4851618 
2.1286416 
-0.9374542 
1.2062176 
1.1970600 
1.8779879 
1.0888278 
1.6379587 
3.2865109 
2.5676486 
2.028 1008 
0.8765109 
1.7695006 
2.0278913 
1.3629063 
1.6232943 
2.4152275 


TABLE 20 
TEST SET ONE 


Y 


3.5978136 
-0.3642989 
1.0218046 
3.2826479 
2.7816672 
1.9686457 
2.6941914 
1.3421858 
1.3385338 
2.7176880 
3.9561078 
3.2294325 
2.0122998 
3.4452995 
2.3519064 
1.9137510 
2.2349597 
2.1070889 
2.5508559 
3.3621478 
1.7760771 
3.2315889 
4.0529999 
3.1099944 
3.7031440 
2.9875884 
5.0984962 
4.0441594 
1.3458853 
3.2349733 
1.4321380 
4.3081925 
3.7146003 
3.9994056 
4.2742129 
5.1924022 
3.1599492 
3.3825862 
4.1757696 
4.5778941 
2.5246523 
3.1299786 
3.7598849 
1.2771573 
4.9586547 
2.8137205 
5.0334870 
3.1335435 
4.5948086 
4.8204097 


X 


101 
102 
103 
104 
105 
106 
107 
108 
109 
110 
111 
112 
113 
114 
115 
116 
117 
118 
119 
120 
121 
122 
123 
124 
125 
126 
127 
128 
129 
130 
131 
132 
138 
134 
135 
136 
137 
138 
139 
140 
141 
142 
143 
144 
145 
146 
147 
148 
149 
150 


150 


Y 


3.8639735 
4.6424739 
3.9925571 
4.7141715 
3.1026557 
3.5086986 
5.6847319 
3.6997858 
4.5971487 
2.8131531 
4.0385152 
3.7093077 
4.2573194 
5.5364895 
5.5778150 
5.8100211 
4.8393109 
5.2195209 
3.7046249 
4.3492568 
6.1103783 
5.7786937 
5.3587051 
3.7126557 
5.3246669 
5.4300705 
4.6748617 
5.4123149 
7.7259102 
4.7421844 
3.029723 
2.6104761 
4.5603033 
4.6852791 
4.2283128 
7.4729300 
6.4484810 
6.1902920 
5.9801460 
2.7272488 
7.3258741 
5.7079245 
4.8179694 
3.7067425 
5.8890863 
7.9270991 
6.2451185 
6.7661055 
5.7754624 
7.0307144 


X 


151 
152 
153 


155 
156 
157 
158 
159 


161 
162 
163 
164 
165 
166 
167 
168 
169 
170 
171 
172 
173 
174 
175 
176 
177 
178 
179 
180 
181 
182 
183 
184 
185 
186 
187 
188 
189 
190 
191 
192 
193 
194 
195 
196 
197 
198 
199 
200 


Y 


4.5123774 
7.0449390 
6.6461625 
5.9755457 
6.3987827 
7.8304268 
6.3121917 
7.2091639 
5.0608475 
7.5825731 
8.2574717 
5.8956979 
5.5093262 
5.5995017 
7.2911596 
5.8813818 
6.5830282 
5.3130510 
7.7908127 
6.0401256 
7.7141272 
7.6411270 
6.7540765 
7.2609074 
6.6775327 
6.6715890 
8.2278953 
7.1614303 
6.0619503 
8.7667237 
6.1915765 
7.7027237 
6.4755850 
7.0483700 
7.7978105 
8.4897860 
7.1392044 
7.8562970 
7.3386392 
7.6166495 
6.4476388 
7.5483537 
9.2075244 
6.9231787 
6.4990181 
9.4141318 
9.0460399 
8.7064297 
6.4983612 
9,5544659 


Y 


0.4570756 
-1.3538641 
2.1577621 
1.6140663 
1.0551468 
1.3229449 
0.2645429 
2.6453778 
0.9949566 
2.7181328 
1.1688202 
1.7586682 
-0.5783523 
0.1460898 
2.0713177 
0.1631187 
1.3549229 
0.3917761 
0.5892597 
-0.6526318 
-0.602987 1 
1.6217221 
0.5609396 
-0.2254246 
-0.2248702 
1.3679551 
0.4151033 
0.9491606 
0.7960204 
1.1185767 
0.7252823 
-1.0546655 
0.6232166 
1.3344005 
0.8986088 
-2.2470305 
-0.1831108 
-0.2721655 
0.3287841 
-0.5403718 
-0.0711898 
1.4975242 
0.6989979 
0.0800240 
-1.1506912 
-0.3364633 
-0.1564088 


-0.8992433 


-0.7161574 
-0.0009194 


TABLE 21 
TEST SET TWO 


M 


1.1056374 
-2.9317810 
-1.6202036 

0.5669487 
-0.0068339 
-0.8917159 
-0.2370383 
-1.6588701 
-1.7312588 
-0.4197057 

0.7522926 
-0.0395823 
-1.3206523 

0.0497107 
-1.1049823 
-1.6030669 
-1.3403846 
-1.5253494 
-1.1372168 
-0.3800745 
-2.0187875 
-0.6143904 

0.1574513 
-0.8335629 
-0.2868485 
-1.0472555 

1.0203924 
-0.0756075 
-2.8139453 
-0.9633214 
-2.8030239 

0.0377556 
-0.5895273 
-0.3368381 
-0.0925853 

0.7965962 
-1.2633354 
-1.0666680 
-0.2979676 

0.0811356 
-1.9936927 
-1.4085479 
-0.7974495 
-3.2976455 

0.3676870 
-1.7921467 

0.4139455 
-1.4984893 
-0.0485762 

0.1667661 


X 


101 

102 
103 
104 
105 
106 
107 
108 
109 
110 
111 

112 
113 
114 
115 
116 
117 
118 
119 
120 
121 
122 
123 
124 
125 
126 
127 
128 
129 
130 
131 

132 
133 
134 
135 
136 
137 
138 
139 
140 
141 

142 
143 
144 
145 
146 
147 
148 
149 
150 


151 


Y 


-0.7988832 
-0.0285995 
-0.6857874 
0.0294491 
-1.5876051 
-1.1863163 
0.9856908 
-1.0026112 
-0.1079922 
-1.8941798 
-0.6705179 
-1.0009952 
-0.4538849 
0.8246897 
0.8656625 
1.0976953 
0.1269276 
0.5071318 
-1.0077820 
-0.3632442 
1.3976432 
1.0655210 
0.6448279 
-1.0022558 
0.6083290 
0.7118525 
-0.0457512 
0.6887321 
2.9987233 
0.0107011 
-1.1073564 
-2.1319047 
-0.1887889 
-0.0714380 
-0.5369943 
2.6980175 
1.6628992 
1.3929299 
1.1698476 
-2.0971853 
2.4860636 
0.8514574 
-0.05647 17 
-1.1870254 
0.9746058 
2.9904894 
1.2849344 
1.7808755 
0.763691 
1.9908847 


X 


151 
152 
153 
154 
155 
156 
157 
158 
159 
160 
161 
162 
163 
164 
165 
166 
167 
168 
169 
170 
171 
172 
173 
174 
175 
176 
177 
178 
179 
180 
181 
182 
183 
184 
185 
186 
187 
188 
189 
190 
191 
192 
193 
194 
195 
196 
197 
198 
199 
200 


Y 


-0.5570467 
1.9443678 
1.5128773 
0.8079680 
1.1953248 
2.5894945 
1.0321866 
1.8884863 

-0.3021013 
2.1757581 
2.8052014 
0.3963923 

-0.0385836 
0.0014325 
1.6413922 
0.1783960 
0.8253250 

-0.5008452 
1.9192740 
0.1095231 
1.7230700 
1.5882570 
0.6380705 
1.0804791 
0.4314349 
0.3586158 
1.8468834 
0.7112616 

-0.4584467 
2.1750750 

-0.4722970 
0.9657036 

-0.3354497 
0.1625056 
0.8363578 
1.4520438 
0.0245292 
0.6641043 
0.0684045 
0.2679094 

-0.9800088 
0.0414590 
1.6211055 

-0.7429779 

-1.2470265 
1.5881130 

1.1400245 

0.7204593 


-1.5674586 


1.4089659 


ү 


0.5178755 
-1.2306658 
2.3449535 
1.8668391 
1.3750802 
1.7116070 
0.7234874 
3.1761424 
1.5990598 
3.3970718 
1.9240685 
2.5916733 
0.3338285 
1.1388347 
3.1459821 
1.3210230 
2.5973502 
1.7199704 
2.0044237 
0.8506615 
0.9895501 
3.3045709 
2.3351194 
1.6410555 
1.7348275 
3.4217348 
2.5637749 
3.1934782 
3.1366808 
3.5562190 
3.2604860 
1.5786193 
3.3550412 
4.1651618 
3.8286416 
0.7825458 
2.9462176 
2.9570600 
3.6579879 
2.8888278 
3.4579587 
5.1265109 
4.4276486 
3.9081008 
2.7765109 
3.6895006 
3.9678913 
3.3229063 
3.6032943 
4.4152275 


TABLE 22 
TEST SET THREE 


Y 


5.4778136 
1.3957011 
2.6618046 
4.8026479 
4.1816672 
3.2486457 
3.8541914 
2.3821858 
2.2585338 
3.5176880 
4.6361078 
3.7894325 
2.4522998 
3.7652995 
2.5519064 
9997910 
2.1949597 
1.9470889 
2.2708559 
2.9621478 
1.2560771 
2.5915889 
3.2929999 
2.2299944 
2.7031440 
1.8675884 
3.8584962 
2.6841594 
-0.1341147 
1.6349733 
-0.2878620 
2.4681925 
1.7546003 
1.9194056 
2.0742129 
2.8724022 
0.7199492 
0.8225862 
1.4957696 
1.7778941 
-0.3953477 
0.0899786 
0.5998849 
-2.0028427 
1.5586547 
-0.7062795 
1.3934870 
-0.6264565 
0.7148086 
0.8204097 


X 


101 
102 
103 
104 
105 
106 
107 
108 
109 
110 
111 
112 
113 
114 
115 
116 
117 
118 
119 
120 
121 
122 
123 
124 
125 
126 
127 
128 
129 
130 
131 
132 
133 
134 
135 
136 
137 
138 
139 
140 
141 
142 
143 
144 
145 
146 
147 
148 
149 
150 


152 


Ү 


-0.1760265 
0.5704739 
-0.1114429 
0.5781715 
-1.0653443 
-0.6913014 
1.4527319 
-0.5642142 
0.3011487 
-1.5148469 
-0.3214848 
-0.6826923 
-0.1666806 
1.0804895 
1.0898 150 
1.2900211 
0.2873109 
0.6355209 
-0.9113751 
-0.2987432 
1.4303783 
1.0666937 
0.6147051 
-1.0633443 
0.5166669 
0.5900705 
-0.1971383 
0.5083149 
2.7899102 
-0.2258156 
-1.3708271 
-2.4215239 
-0.5036967 
-0.4107209 
-0.8996872 
2.3129300 
1.25648 10 
0.9662920 
0.7241460 
-2.5607512 
2.0058741 
0.3559245 
-0.5660306 
-1.7092575 
0.4410863 
2.4470991 
0.7331185 
1.2221055 
0.1994624 
1.4227144 


X 


151 
152 
153 


155 
156 
157 
158 


160 
161 

162 
163 
164 
165 
166 
167 
168 
169 
170 
171 

172 
173 
174 
175 
176 
177 
178 
179 
180 
181 

182 
183 
184 
185 
186 
187 
188 
189 
190 
191 

192 
193 
194 
195 
196 
197 
198 
199 
200 


Үү 


-1.1356226 
1.3569390 
0.9181625 
0.2075457 
0.5907827 
1.9824268 
0.4241917 
1.2811639 

-0.9071525 
1.5745731 
2.2094717 

-0.1923021 

-0.6186738 

-0.5684983 
1.0831596 

-0.3666182 
0.2950282 

-1.0149490 
1.4228127 

-0.3678744 
1.2661272 
1.1531270 
0.2260765 
0.6929074 
0.0695327 
0.0235890 
1.5398953 
0.4334303 

-0.7060497 
1.9587237 

-0.6564235 
0.8147237 

-0.4524144 
0.0803703 
0.7898105 
1.4417860 
0.0512044 
0.7282970 
0.1706392 
0.4086495 

-0.8003612 
0.2603537 
1.8795244 

-0.4448213 

-0.9089819 
1.9661318 
1.5580399 
1.1784297 

-1.0696388 
1.9464659 


APPENDIX E 
SAMPLE SESSION USING SUPSMO PROGRAM 


The following is a computer session showing the interaction between 
the smoothing program SUPSMO and a user. The steps necessary to 
run the smoothing program SPTLIN are very similar to those in this 
session. It should be remembered that SPTLIN requires a terminal 


which can access 2M of computer storage memory. 
зирзпо 


YOU р INITIATED AN ALGORITHM TO SMOOTH A SET OF DATA 
NUI. "SUPER SMOOTHER" DEVELOPED BY FRIEDMAN AND 
ENEDUNEORD UNIVERSITY DEPT. OF STATISTICS 

IF GRAPHICS WILL NOT BE USED DEFINE STORAGE AS 1024K 

BY ENTERING 'DEF STOR 1024К! 

ШИЕЛОМЕП) BY ‘I CMS", 

THEN BY 'SUPSMO! 

DO YOU WISH TO CONTINUE? 

ENCER Y FOR YES OR ANY OTHER KEY TO EXIT: 


у 


IN ORDER TO USE THIS ALGORITHM YOU MUST HAVE ON HAND THE 
FOLLOWING: 


1. FILENAME OF DATA FILE (FILETYPE DATA) WITH DATA TO BE 
SMOOTHED 


E IF DATA POINTS ARE NOT IN CHRONOLOCICAL ORDER, YOU NEED 
En A FILE (FILETYPE ORDER) WITH INDICES CORRESPONDING TO 
POINTS INDICATING THE ORDER OF THE DATA POINTS. 

"DM E OF DATA FILE WHERE SMOOTHED OUTPUT WILL BE 
OR IF YOU WANT TO WRITE OUTPUT INTO APL HAVE ON HAND 
THE VARIABLE AND WORKSPACE NAMES THAT WILL STORE THE 

OUTPUT. 

ELE YOU WANT TO SMOOTH THE DATA USING ONLY ONE WINDOW 
HAVÉ ON HAND THE DECIMAL FRACTION OF THE DATA TO BE USED. 
5. ТЕ YOU WANT TO SMOOTH THE DATA USING THREE WINDOW SIZES 

о ON HAND THE THREE DECIMAL FRACTIONS OF THE DATA TO BE 


DO YOU WISH TO CONTINUE? 
ENTER Y FOR YES OR ANY OTHER KEY TO EXIT: 


153 


У 


ENTER FILENAME OF FILE WHICH CONTAINS THE DATA ТО ВЕ 
SMOOTHED: 


water 


ARE DATA POINTS TO BE SMOOTHED IN CHRONOLOGICAL ORDER? 
ENTER Y FOR YES OR N FOR М0: 


У 


ТНЕ DATA YOU WANT ТО SMOOTH IS IN WATER DATA 

WHERE DO YOU WANT TO WRITE THE SMOOTHED OUTPUT? CMS OR APL? 
YOU CAN PLOT THE SMOOTHED OUTPUT IF YOU ARE LOGGED ON 
Ол Ни THAT САМ ACCESS CRAFSTAT, I.E. HAVE 2M OF 
BUT THE QUIPUT MUST BE STORED IN AN APL VARIABLE 

ENTER АРЫ бе СМ; 


ар1 


NOT USING THE NAME OF THE FILE WITH THE INPUT DATA, WATER 
c THE NAME OF THE APL VARIABLE THAT WILL STORE THE 


smuf ig53 


DO YOU WANT TO PLOT THE OUTPUT? 
ENTER Y FOR YES OR N FOR NO: 


22 


CAN YOU ACCESS 2M qi STORAGE ON THIS DISK (TERMINAL)? 
ENTER Y FOR YES OR N FOR NO: 


Dd 


PLEASE READ THE FOLLOWING INSTRUCTIONS VERY CAREFULLY 
ARE YOU READY TO START THE SUPER SMOOTHING PROGRAM? 
ENTER Y ВОН YES OR ANY OTHER KES ЦД. 


7 


xxxxx PLEASE WAIT THE SMOOTHING PROGRAM 15 BEING 
СОМРТГЕ) хжхжххккккх 


VS FORTRAN COMPILER ENTERED. 1621230 
**MAIN**x END OF COMPILATION 1 *x*x*x**xx 
**SUPSMU** END OF COMPILATION 2 жжжжжж 
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*xSMOOTH*x*x END OF COMPILATION 3 жжжжжж 
**xSORTER*x* END OF COMPILATION Чч xxxxxx 
*xxBLKDT#xx END OF COMPILATION 5 xxxxxx 
VS FORTRAN COMPILER EXITED. 16:11:36 


x***x*x*PLEASE WAIT SMOOTHING PROCRAM Э, ВЕІМС 


ГгОДРЕржжхжхжжжхжжхжхжжжжжж ж 


EXECUTION BEGINS . 
eae THE NUMBER OF DATA POINTS TO BE SMOOTHED---INTEGER 


671 


ARE THE INPUT DATA POINTS IN CHRONOLOGICAL ORDER? 
ENTER O FOR NO OR 1 FOR YES 


ENTER 1.0 IF YOU DESIRE TO USE ONLY ONE SPAN VALUE 
ENTER 0.0 IF YOU DESIRE TO USE THREE SPAN VALUES 


0.0 


ENTER THE LOWEST SPAN VALUE: 
FRACTION OF 671 Т.Е. A REAL NUMBER BETWEEN 0.0 AND 1.0 


0.00745 


ENTER THE MIDDLE SPAN VALUE: 
FRACTION OF 671 Т.Е. A REAL NUMBER BETWEEN 0.0 AND 1.0 


09016393 


ENTER THE HICHEST SPAN VALUE: 
FRACTION OF 671 Т.Е. А REAL NUMBER BETWEEN 0.0 AND 1.0 


0175 


ТЕ ОМЕ d THE SPAN VALUES IS SMALL 

Iu RESULTS IN A SMALL WINDOW SIZE (10 OR LESS) 
YOU MAY WIS TO ADJUST THE ROBUSTNESS 

BY ENTERING A REAL NUMBER СТ 0.0 ВОТ LT 10.0 

OR FOR NO ROBUST ADJUSTMENT ENTER 0.0 

ENTER YOUR CHOICE 


956 
xxxxx PLEASE WAIT SMOOTHING PROGRAM NOW RUNNING*xx*xxx 
*x***PLEASE WAIT, LINKINC TO GRAFSTATRKKkKKKKK KK KK KKK KKK 
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193) R/O 

194) R/O 

391) R/O 
VS 


CLEAR WS 
SAVED 06:26:45 cm 
WSSIZE IS 1188956 


CMS MATRIX IS NOT RECTANGULAR. ROW ONE HAS 5 ELEMENTS 
ROW 135 HAS 1 ELEMENT(S) 
INFORMATION TRANSFER HAS STOPPED AT THIS LINE 


16:12:42 09/04/85 SUPSMO 
SAVED 15:18:53 057097 eo 
WSSIZE 15 1188956 


а 4/01/85 RELEASE OF GRAFSTAT. ІТ RUNS ON THE 
В CONTROL VECTORS FROM EARLIER RELEASES WILL 
TO RUN. ТЕ YOU )COPY RATHER THAN )LOAD THIS WORKSPACE YOU 


UST 

EXECUTE IHE FUNCITONE TENZ BEFORE STARTING. THE NEXT. 
RELEASE IS 

SCHEDULED FOR 9/85. 


TO BEGIN, TYPE: START 
FOR MORE INFORMATION, TYPE: DESCRIBE 


NOT COPIED? RCODE GET XBLANKS VCAT 

SAVED 16:12:42 09/04/85 

ТНЕЕМТІБЕ DATA FILE THAT YOU WANTED SMOOTHED HAS BEEN 
TRANSFERRED 

TO THIS WORKSPACE SO THAT YOU MAY BE ABLE TO PLOT BOTH 

THE SMOOTHED AND UNSMOOTHED DATA. 

ci UNSMOOTHED DATA IS IN THE VARIABLE WITH THE SAME NAME 


THE DATA FILE THAT YOU HAVE YOUR INPUT DATA IN. 


P L 5,0 


DO YOU WISH TO GO_INTO APL OR CONTINUE? 
ENTER O FOR APL OR 1 FOR CONTINUE М: 


YOU HAVE ACTIVATED THE PLOTTING FUNCTION 
BI RE THAT THE USER IS FAMILIAR WITH THE CRAFSTAT 
AND THE AXIS CONTROL FUNCTION 

IF YOU RECEIVE (MAKE) AN ERROR MESSAGE DO THE FOLLOWING 

1. ENSURE THAT VM READ IS DISPLAYED IN LOWER RIGHT CORNER 
OF SCREEN 

2. PRESS THE ENTER KEY 

3e ENTER P AG 
„20 лог A LETTER HOLD THE APL/ALT KEY DOWN AND PRESS 
THE PLOTTING FUNCTION WILL RESTART AT THE BEGINNING 
Еа FUNCTION CAN BE EXITED АТ ANY INPUT POINT BY 


=» 
AT ANYTIME THAT YOU EXIT THE PLOTTING FUNCTION 
YOU WILL BE IN THE GRAFSTAT WORKSPACE 
IF YOU WISH TO RETURN TO CMS ENTER 
JOFF HOLD 
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ENTER X VARIABLE(S) (ENCLOSED IN QUOTES), IF ENTERING MORE 
THAN ONE VARIABLE 

SEPARATE VARIABLES ща SEMICOLON AND USE QUOTES 

Emo ТА ОЕ 'Х1;Х2' N 


'( 670)+59! 


E К Ан и (ENCLOSED IN QUOTES AND MUST BE OF SAME 
IF ENTERING MORE THAN ONE VARIABLE, SEPARATE WITH SEMICOLON 
т n oon DC Von donus ENCLOSING ENTIRE STRING 


'670 WATER; SMUFICGS3'! 


ENTER A VECTOR INDICATING TYPE(S) OF PLOT; O=SYM ОМГУ; 
1=LINE ONLY 
ЖЭО ОЕ 1 ОК 0 1 ОЕ 0 O OR 1 O OR 1 1 N 


ENTER ТҮРЕ ОЕ SYMBOL CORRESPONDING TO EACH SYMBOLS ONLY 
PLOT (IN QUOTES) 
emcee '.' OR '.х! YOU CAN USE .х+ М 


Дан EINE ет INDICATING TYPE(S) OF LINES; 1=SOLID LINE; 
mc. OR 3 OR 1 3 OR ANY OTHER COMBINATION OR LINE TYPES 
IN CRAPSTAT N : 

1 


ENTER SCALE OF X-AXIS (IN QUOTES) OR P (IN QUOTES) FOR 
PREVIOUS SCALE 
eee 'LIN' OR 'LIN XMIN XMAX' OR 'P' N 


Piva. 735 


ENTER SCALE OF Y-AXIS (IN QUOTES) OR P (IN QUOTES) FOR 
PREVIOUS SCALE 
КОСО ЕЛМ OR 'LIN YMIN YMAX' OR 'P' М 


цели вал" 


ENTER THE PLOT HEADER с QUOTES) OR EMPTY QUOTES 
В !ТІТСЕ! ОК N 


' SMOOTHING WITH 7SUPERSMOOTHER, ALPHA= 0.0% SPAN(S)= 
ВОН, 0.016393, 0.0 
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ENTER X-AXIS LABEL (IN QUOTES) OR 
A PAIR OR EMETY TUE M OR NO LABEL OR TO USE AXIS CONTROL 
Е.С. VLABETLUS DM 


' JULIAN CALENDAR ПАТЕ! 


ENTER Y- AXIS LABEL (IN QUOTES) OR 
A PAIR OF EMPTY ще OR NO LABEL OR TO USE AXIS CONTROL 
БОСНА ВА КОЕ N 


'TEMPERATURE IN DEGREES CENT.' 


DO YOU WANT TO RUN THIS PAGE? 
ENTER 0 FOR NO OR 1 FOR YES N : 


0 


DO YOU WANT TO EXIT THIS FUNCTION? 
ENTER O FOR NO OR 1 FOR YES 


RES 


DO YOU WANT TO RESTART THIS пещери ще 
ЕМТЕК 0 FOR NO OR 1 FOR YES N 


THE ONLY THING LEET TO DO IS TRE AXIO ONTROL 

WITH THE PARTIAL PLOT TRAT YOU HAVE JUST —EIMISHEN 
CONSTRUCTING 

ENTER A 3 ELEMENT VECTOR FOR PARTIAL PLOT 

и 100): LINES AND SYMBOLS ARE (NOT) SHOWN ON 


к; 1(0): HEADER AND AXES ARE (NOT) SHOWN ON 
р СВТ, 1(0): AXES, GRIDS, AND GRID LINES ARE (NOT) 


E.G. 1 1 0 WILL SHOW EVERYTHING ON GRAPH EXCEPT AXES AND 
CRID LINES N : 


LII 9 
ENTER A "M ELEMENT ВСЕ POR AXES AND GRID a 
157 ELEMENT, X-AXIS: Гг 2 ТОР, а = о 
2ND ELEMENT, Y-AXIS: LEF = RIGHT, On 0 
,3RD, ELEMENT, VERTICAL GRID LINES: ZNO CRIDI TSDOTTED E 


„ЧТ ELEMENT, HORIZON. GRID LINES: O=NO GRID, 1=DOTTED, OR 
E.G. 2 12 2 WILL DISPLAY AXIS AT TOP AND LEFT AND SOLID 
GRID LINES Q: 

0100 
PLEASE WAIT RUNNING PAGE 
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ENTER X-AXIS TIC MARKS а о VECTOR 
OR ENTER O FOR STANDARD TIC MARKS 

OR ENTER 1 FOR NO TIC MARKS 

ШИ 1 5 11 ОД A VECTOR NAME OR O OR 1 

VTL SHOW TIC MARKS AT X=1, X=5, AND Х-11 8: 


SeS O S Тан 2412 


31 5 2 304 334 365 396 
424 T T35 516546577608 633 669 6 


ENTER X-AXIS SYMBOLS (IN QUOTES) 

OR ENTER O WITHOUT QUOTES FOR STANDARD SYMBOLS 
OR ENTER 1 WITHOUT QUOTES FOR NO SYMBOLS 

Е.С. '1970;1971' OR A VECTOR NAME OR O OR 11 Ñ 


Мо антене а 3 55: 2080: 2172: 2264:2355' 


ENTER X-AXIS SYMBOLS LOCATIONS VECTOR 

OR ген. 0 ара ТОР | M auc LOCATIONS OR NO SYMBOLS 
nec. 6 OR A VECTOR NAME OR 

6 18 WILE SHOW 1970 AT je 6 AND. 1971 A7 Xz18 N 


ВОВА 702 261 3557445 536 629 721 


ENTER Y-AXIS TIC MARKS LOCATION VECTOR 
OR ENTER 0 FOR STANDARD TIC MARKS 

OR pees 1 ЕОК МО ТЇС MARKS 

Е.С. 0 1 OR A VECTOR а ОК е OR 1 
ТТС harks Af Y2 1, Y=0, AND У=1 М 


101 WILL SHOW 


0 


ENTER Y-AXIS SYMBOLS (IN QUOTES) 

OR ENTER O WITHOUT QUOTES FOR STANDARD SYMBOLS 
OR ENTER 1 WITHOUT QUOTES FOR NO SYMBOLS 

feo. ГО МІР HI' OR VECTOR NAME OF O OR 1 N 


ENTER Y-AXIS SYMBOLS LOCATIONS VECTOR 

OR ENTER O FOR SYMBOLS AT DEFAULT LOCATIONS OR NO SYMBOLS 
m 101 OR а еа драма OR O 101 WILL SHOW LO AT Y= 
I» AT Yso, HI AT N 


THESE AXIS CONTROL ENTRIES WILL NOW BE RUN 
DO YOU WANT TO RERUN THE PLOT INPUTS YOU о 


BEFORE RUNNING THIS AXIS CONTROL FUNCTION 
КОЛЛЕК О FOR NO OR 1 FOR YES N : 
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DO YOU WANT TO RERUN THE PLOT INPUTS YOU ENTERED 
BEFORE RUNNING THIS AXIS CONTROL FUNCTION? 
ENTER 0 FOR NO OR 1 FOR YES Ы: 


WITH THE PARTIAL PLOT THAT YOU HAVE JUST FINISHED 
CONSTRUCTING 
TER A 3 ELEMENT VECTOR FOR PARTIAL PL 


EN OT 
ee a 1(0): LINES AND SYMBOLS ARE (NOT) SHOWN ON 


ERR 1(0): HEADER AND AXES ARE (NOT) SHOWN ON 
Soa T TERREN 1(0): AXES, GRIDS, AND GRID LINES ARE (NOT) 


E.G. 1 1 0 WILL SHOW EVERYTHING ON GRAPH EXCEPT AXES AND 
GRID LINES N : 


00 0 


ENTER A 4 ELEMENT VECTOR FOR AXES AND GRID Сари 

157 ELEMENT, X-AXIS: 0 BOTTOM, 2 = TOP О а ЕЕ 
2ND ELEMENT, Y-AXIS: 1 = LEFT, 3 = nro ы OR 21 2 02 m 
„3BD ELEMENT, VERTICAL GRID LINES: O=NO GRID, 1=DOTTED, OR 


UTH ELEMENT, HORIZON. GRID LINES: O=NO GRID, 1-ПОТТЕрР, ОС 


AN E Щи 
E 2 2 WILL DISPLAY AXIS AT TOP AND LEFT AND SOLID 
CRID ' LÉNES М : 


созт 
PLEASE WAIT RUNNING PAGE 


ENTER X-AXIS TIC MARKS LOCATION VECTOR 

OR ENTER O FOR STANDARD TIC MARKS 

OR ENTER 1 FOR NO TIC MARKS 

E.G. 15 11 OR A VECTOR NAME OR O OR 1 

1 5 11 WiLL SHOW TiC MARKS AT X21, 5, А. ОИ 


90 170 260 3250 U0 4530 62057 bo 


ENTER X-AXIS SYMBOLS (IN QUOTES) 

OR ENTER O WITHOUT QUOTES FOR STANDARD SYMBOLS 
OR ENTER 1 WITHOUT QUOTES FOR NO SYMBOLS 

Е.С. '1970;:1971' OR A VECTOR NAME TORK OFORE TENE: 


ENTER X-AXIS SYMBOLS LOCATIONS VECTOR 

OR ENTER O FOR SYMBOLS AT DEFAULT LOCATIONS OR NO SYMBOLS 
E.G. 6 18 OR A VECTOR NAME OR O 

6 18 WILL SHOW 1970 AT Xs6 AND 1971 A7 Xz18 N : 
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ENTER Y-AXIS TIC MARKS LOCATION VECTOR 

OR ENTER O FOR STANDARD TIC MARKS 

OR ШІГЕН 1 FOR NO TIC MARKS E 

Сс. O 1 ОЕ А VECTOR R OR е ОН. 10 1 WILL SHOW 
ZIG harks ДЕ а ЕОР = Т 


1 


МЕДЕН Y- а SYMBOLS (IN QUOTES) 

OR ENTER WITHOUT QUOTES FOR SPANDARD SYMBOLS 
OR ENTER 1 WITHOUT QUOTES FOR NO SYMBOLS 

М ТБО МІР НІ! ОК VECTOR NAME OR O OR 1 N 


ENTER Y-AXIS SYMBOLS LOCATIONS VECTOR 

OR 45 O FOR SYMBOLS AT DEFAULT. LOCATIONS OR NO SYMBOLS 
шие. On OR VEETOR NAME OR O 101 WILL SHOW LO AT Y= 
1, MID Ar мал УЕ М 


0 


THESE AXIS CONTROL ENTRIES WILL NOW BE RUN 
EXOU JANT IO RERUN THE PLOT INPUTS YOU ми 
BEFORE RUNNING THIS AXIS CONTROL FUNCTION 

ENTER 0 FOR NO OR 1 FOR YES NM : 


0 


DO YOU WANT TO DO ANOTHER =; CONTROL PAGE? 
ENTER О FOR NO OR 1 FOR YES N 


0 


DO YOU WANT TO RESTART THE FUNCTION? 

TE YOU DO NOT YOU WILL BRIT THIS FUNCTION 

IF YOU EXIT THIS FUNCTION AND WANT TO RETAIN THIS WORK 

USE THE KEEP FUNCTION AND THEN YOU CAN RETURN TO CMS 

BY ENTERING )OFF HOLD 

IF YOU WANT TO RETURN TO CMS, SIMPLY ENTER )OFF HOLD AFTER 


ВАША 
ENTER O FOR EXIT OR 1 FOR RESTART М: 
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10. 


di 


12. 


Ds 
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